Jul 06 22:46:25.508395 ip-10-0-128-38 systemd[1]: Starting Kubernetes Kubelet... Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.919853 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924186 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924206 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924211 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924216 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924221 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924228 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924233 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:46:25.997390 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924238 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924244 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924255 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924260 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924271 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924282 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924287 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924292 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924296 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924300 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924307 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924314 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924320 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924325 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924339 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924344 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924349 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924353 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924358 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:46:26.249169 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924362 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:46:26.005388 ip-10-0-128-38 systemd[1]: Started Kubernetes Kubelet. Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924366 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924371 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924375 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924381 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924388 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924393 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924402 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924407 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924413 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924419 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924425 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924430 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924458 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.924896 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925018 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925023 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925026 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925033 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925037 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:46:26.261176 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925039 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925042 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925045 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925048 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925050 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925053 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925056 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925059 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925061 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925065 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925067 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925070 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925073 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925076 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925079 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925083 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925086 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925089 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925108 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925132 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:46:26.262393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925144 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925149 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925154 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925158 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925162 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925166 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925170 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925175 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925179 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925183 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925188 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925191 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925195 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925199 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925204 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925208 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925214 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925218 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925221 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925224 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:46:26.263478 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925227 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925229 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925232 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925235 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925237 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925240 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925243 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925246 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925249 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925253 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925256 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925260 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.925262 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926023 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926030 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926033 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926038 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926041 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926044 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:46:26.264614 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926047 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926049 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926052 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926055 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926058 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926063 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926066 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926068 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926071 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926074 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926077 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926079 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926087 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926105 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926112 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926117 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926120 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926123 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926127 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:46:26.265522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926129 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926134 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926138 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926141 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926144 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926146 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926149 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926152 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926155 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926158 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926160 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926163 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926166 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926169 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926172 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926177 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926181 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926184 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926188 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926191 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:46:26.266430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926194 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926196 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926199 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926201 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926203 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926206 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926209 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926212 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926214 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926217 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926220 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926223 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926225 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926228 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926230 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926233 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926235 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926237 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926240 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:46:26.267531 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926243 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926246 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926248 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926251 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926254 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926257 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926259 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926262 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926264 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926266 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926269 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926273 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926275 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926278 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926282 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926285 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926289 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926291 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926294 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926298 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:46:26.268414 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926301 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926303 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926306 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926308 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926311 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926313 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926316 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926318 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926321 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926323 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926326 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926328 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926331 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926333 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926336 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926740 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926744 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926747 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926750 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926753 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:46:26.269513 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926755 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926758 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926760 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926765 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926767 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926770 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926773 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926775 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926782 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926784 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926787 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926790 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926793 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926796 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926798 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926801 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926803 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926806 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926808 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926811 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:46:26.270405 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926814 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926816 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926819 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926821 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926823 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926826 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926828 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926831 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926833 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926836 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926838 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926841 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926843 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926845 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926848 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926852 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926855 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926857 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926860 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926863 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:46:26.271346 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926867 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926870 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926872 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926875 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926878 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926880 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926882 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926885 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926888 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926892 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926894 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926897 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926899 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926902 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926904 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926907 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926909 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926912 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926914 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:46:26.272260 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926917 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926919 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926923 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926927 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926930 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926933 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926936 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926939 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926943 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926945 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926949 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926951 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926954 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926957 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926960 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926962 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926965 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926968 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926970 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:46:26.273158 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926973 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926975 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926978 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926980 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926983 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926985 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926988 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926990 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926994 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.926997 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927000 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927002 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927005 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927008 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927010 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927012 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927107 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927118 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927127 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927134 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927139 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 22:46:26.273979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927144 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927149 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927153 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927156 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927159 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927164 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927168 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927171 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927174 2561 flags.go:64] FLAG: --cgroup-root="" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927177 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927180 2561 flags.go:64] FLAG: --client-ca-file="" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927184 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927187 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927192 2561 flags.go:64] FLAG: --cluster-domain="" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927195 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927198 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927201 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927205 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927208 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927211 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927214 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927217 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927220 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927223 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 22:46:26.274877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927226 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927230 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927232 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927236 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927239 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927242 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927245 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927249 2561 flags.go:64] FLAG: --enable-server="true" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927253 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927257 2561 flags.go:64] FLAG: --event-burst="100" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927260 2561 flags.go:64] FLAG: --event-qps="50" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927263 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927266 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927272 2561 flags.go:64] FLAG: --eviction-hard="" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927275 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927278 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927281 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927284 2561 flags.go:64] FLAG: --eviction-soft="" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927287 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927290 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927294 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927297 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927300 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927303 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927306 2561 flags.go:64] FLAG: --feature-gates="" Jul 06 22:46:26.275973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927310 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927312 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927315 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927318 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927322 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927325 2561 flags.go:64] FLAG: --help="false" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927328 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927331 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927334 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927337 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927340 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927344 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927347 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927349 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927352 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927356 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927359 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927362 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927365 2561 flags.go:64] FLAG: --kube-reserved="" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927368 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927373 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927376 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927379 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927382 2561 flags.go:64] FLAG: --lock-file="" Jul 06 22:46:26.277101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927384 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927388 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927391 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927397 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927400 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927403 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927406 2561 flags.go:64] FLAG: --logging-format="text" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927409 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927412 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927415 2561 flags.go:64] FLAG: --manifest-url="" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927418 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927422 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927425 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927429 2561 flags.go:64] FLAG: --max-pods="110" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927432 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927435 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927437 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927440 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927443 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927446 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927449 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927456 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927459 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927463 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 22:46:26.278419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927467 2561 flags.go:64] FLAG: --pod-cidr="" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927469 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927475 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927478 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927483 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927486 2561 flags.go:64] FLAG: --port="10250" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927489 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927492 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e4254e61a4e4aa1e" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927495 2561 flags.go:64] FLAG: --qos-reserved="" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927498 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927501 2561 flags.go:64] FLAG: --register-node="true" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927504 2561 flags.go:64] FLAG: --register-with-taints="" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927508 2561 flags.go:64] FLAG: --registry-burst="10" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927511 2561 flags.go:64] FLAG: --registry-qps="5" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927514 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927517 2561 flags.go:64] FLAG: --reserved-memory="" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927521 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927524 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927527 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927530 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927532 2561 flags.go:64] FLAG: --runonce="false" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927535 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927538 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927541 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927544 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 22:46:26.279517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927547 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927550 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927553 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927556 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927559 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927561 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927564 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927569 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927572 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927575 2561 flags.go:64] FLAG: --system-cgroups="" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927578 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927585 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927588 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927590 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927595 2561 flags.go:64] FLAG: --tls-min-version="" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927598 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927601 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927603 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927607 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927610 2561 flags.go:64] FLAG: --v="2" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927614 2561 flags.go:64] FLAG: --version="false" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927618 2561 flags.go:64] FLAG: --vmodule="" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927622 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.927625 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927742 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:46:26.282983 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927746 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927751 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927755 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927758 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927761 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927764 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927766 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927769 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927771 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927774 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927777 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927779 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927783 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927786 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927791 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927794 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927796 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927799 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927806 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:46:26.284138 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927809 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927812 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927815 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927817 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927820 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927823 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927825 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927828 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927831 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927833 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927836 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927839 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927842 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927844 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927847 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927849 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927851 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927854 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927857 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:46:26.290975 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927859 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927862 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927865 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927867 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927870 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927872 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927875 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927877 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927881 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927884 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927887 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927889 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927893 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927896 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927899 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927902 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927905 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927909 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927912 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:46:26.291617 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927915 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927918 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927921 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927924 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927926 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927929 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927932 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927935 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927937 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927939 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927942 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927944 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927947 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927949 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927952 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927954 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927957 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927959 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927962 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927964 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:46:26.292225 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927967 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927970 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927973 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927975 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927978 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927982 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927984 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927987 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927989 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927992 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927994 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927997 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.927999 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928002 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928004 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928007 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928009 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928012 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928015 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:46:26.292853 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928018 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.928020 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.928845 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.935710 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.935728 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.935747 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.935753 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935814 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935819 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935823 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935826 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935828 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935831 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935834 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935837 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:46:26.293502 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935839 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935843 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935845 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935848 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935852 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935859 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935862 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935865 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935868 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935870 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935874 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935876 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935879 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935881 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935884 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935887 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935891 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935894 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935897 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:46:26.310113 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935900 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935903 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935906 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935908 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935911 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935913 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935916 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935918 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935921 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935924 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935926 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935928 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935931 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935933 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935936 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935938 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935941 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935943 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935948 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:46:26.312277 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935952 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935955 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935958 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935961 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935964 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935967 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935970 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935973 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935975 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935978 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935980 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935984 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935987 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935989 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935992 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935995 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.935998 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936000 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936003 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936005 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:46:26.314753 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936008 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936010 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936013 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936015 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936018 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936020 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936023 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936025 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936028 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936030 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936033 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936037 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936039 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936042 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936045 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936047 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936050 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936052 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936055 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936057 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:46:26.317308 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936060 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936062 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936065 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936068 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936071 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936075 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936080 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936083 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936086 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936102 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936106 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936108 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936111 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.936117 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936255 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:46:26.318119 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936261 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936264 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936267 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936270 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936272 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936275 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936278 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936281 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936284 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936288 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936291 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936294 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936296 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936299 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936302 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936304 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936307 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936309 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936312 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936314 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:46:26.320069 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936317 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936319 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936322 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936326 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936329 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936333 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936337 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936340 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936343 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936345 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936348 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936351 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936353 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936355 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936358 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936360 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936363 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936365 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:46:26.321758 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936368 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936370 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936373 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936377 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936380 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936382 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936385 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936388 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936390 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936392 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936395 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936399 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936402 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936404 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936407 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936409 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936412 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936418 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936420 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936423 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:46:26.322648 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936425 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936428 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936431 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936433 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936436 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936438 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936441 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936443 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936446 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936448 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936452 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936455 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936459 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936462 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936464 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936468 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936471 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936474 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936477 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:46:26.323443 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936480 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936482 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936485 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936487 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936490 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936493 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936495 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936498 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936500 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936503 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936506 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936510 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936512 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936515 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936517 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936520 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936522 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936525 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936527 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936530 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:46:26.324112 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:25.936532 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.936537 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.936655 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.938691 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.940367 2561 server.go:1019] "Starting client certificate rotation" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.940470 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.940504 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.944469 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.945705 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.959517 2561 log.go:25] "Validated CRI v1 runtime API" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.961672 2561 log.go:25] "Validated CRI v1 image API" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.964898 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 22:46:26.324765 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.968844 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:46:26.325338 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.969723 2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9b2e8aeb-e323-4851-87e7-62203bb0c8d7:/dev/nvme0n1p4 df2658f6-b2e5-44f1-aa7d-b6577dd93f26:/dev/nvme0n1p3] Jul 06 22:46:26.325338 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.969744 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.975898 2561 manager.go:216] Machine: {Timestamp:2026-07-06 22:46:25.973865382 +0000 UTC m=+0.339159038 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096933 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec287bc99fe5fe023b1f0a1b709dabf6 SystemUUID:ec287bc9-9fe5-fe02-3b1f-0a1b709dabf6 BootID:14eaaa59-5a43-4d36-95b1-4dc5db3d4e68 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:bb:cf:6f:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:bb:cf:6f:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:9e:ab:33:90:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.976025 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.976133 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.980324 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.980363 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-38.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.981123 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.981135 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.981144 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.981709 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.983254 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987179 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987327 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987358 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987374 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987388 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987398 2561 kubelet.go:398] "Adding apiserver pod source" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987406 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 22:46:26.325740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987806 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.987823 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.989972 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.991284 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jlj94" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.993014 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.993043 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994594 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994610 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994616 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994622 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994630 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994635 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994644 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994649 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994656 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994662 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994682 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.994688 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.996127 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.996137 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:25.997620 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:25.997652 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-38.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.998209 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jlj94" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.999663 2561 server.go:1268] "Started kubelet" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.999742 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:25.999799 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.004416 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.005346 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-38.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.005731 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.007068 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.010486 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.014509 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.014517 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.014574 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.015021 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.015028 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.015045 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.015168 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.015179 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016203 2561 factory.go:55] Registering systemd factory Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016216 2561 factory.go:223] Registration of the systemd container factory successfully Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016397 2561 factory.go:153] Registering CRI-O factory Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016410 2561 factory.go:223] Registration of the crio container factory successfully Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016476 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016495 2561 factory.go:103] Registering Raw factory Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016508 2561 manager.go:1196] Started watching for new ooms in manager Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016861 2561 manager.go:349] Starting recovery of all containers Jul 06 22:46:26.326362 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.016908 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.020257 2561 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-38.ec2.internal\" not found" node="ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.027531 2561 manager.go:354] Recovery completed Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.033054 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.034995 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.035025 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.035039 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.035722 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.035732 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.035732 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.035798 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.036522 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.036546 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.036559 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.038064 2561 policy_none.go:49] "None policy: Start" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.038077 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.038088 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.039987 2561 policy_none.go:47] "Start" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.115050 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.118793 2561 manager.go:337] "Starting Device Plugin manager" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.118825 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.118835 2561 server.go:87] "Starting device plugin registration server" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.119170 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.119184 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.119286 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.119391 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.119400 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.121541 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.121588 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.146439 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.147836 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.147877 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.147901 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.147940 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.150261 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.219435 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.221445 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.221475 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.221485 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.221508 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.231142 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.231165 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-38.ec2.internal\": node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.245655 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.248861 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal"] Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.248922 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.252476 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:46:26.329132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.252512 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.252533 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.255149 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.255344 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.255378 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.256013 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.256022 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.256041 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.256047 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.256063 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.256052 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.258911 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.258940 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.259692 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.259714 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.259725 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.316952 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.316986 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.330400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.317007 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/301ed78f06a8f545f44b18fb2b42ca58-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.ec2.internal\" (UID: \"301ed78f06a8f545f44b18fb2b42ca58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.345801 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.346050 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-38.ec2.internal\" not found" node="ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.351839 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-38.ec2.internal\" not found" node="ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.417863 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.417901 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.417934 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/301ed78f06a8f545f44b18fb2b42ca58-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.ec2.internal\" (UID: \"301ed78f06a8f545f44b18fb2b42ca58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.417954 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.417980 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/301ed78f06a8f545f44b18fb2b42ca58-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.ec2.internal\" (UID: \"301ed78f06a8f545f44b18fb2b42ca58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.440720 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.418034 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.559616 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.446070 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.559616 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.546683 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.690974 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.647733 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:26.690974 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.648821 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:26.690974 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:26.655806 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 06 22:46:27.928674 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:26.748782 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:27.928674 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:27.914938 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:28.107906 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.005407 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 22:46:28.107906 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.005562 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 22:46:28.107906 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.005570 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 22:46:28.107906 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:28.015712 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:28.107906 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.033559 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:46:28.107906 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.071800 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 22:41:27 +0000 UTC" deadline="2028-01-30 17:53:24.265343817 +0000 UTC" Jul 06 22:46:28.107906 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.071839 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13747h6m56.193508454s" Jul 06 22:46:28.266066 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:28.116608 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:28.266066 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:28.217173 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 06 22:46:28.398834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.294230 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:46:28.398834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.380736 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 06 22:46:28.398834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.391043 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:46:28.398834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.391819 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 06 22:46:28.400527 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.400423 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:46:28.522918 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:28.446319 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301ed78f06a8f545f44b18fb2b42ca58.slice/crio-eded05ba5ee28d80d2e958409a9c98cfa6c24a7e99193dea896b2f3e6ad2ea39 WatchSource:0}: Error finding container eded05ba5ee28d80d2e958409a9c98cfa6c24a7e99193dea896b2f3e6ad2ea39: Status 404 returned error can't find the container with id eded05ba5ee28d80d2e958409a9c98cfa6c24a7e99193dea896b2f3e6ad2ea39 Jul 06 22:46:28.522918 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:28.447045 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc2ad129405fbfb160c1db3475eb64b.slice/crio-140952d1141cd425341b7f8b75e175ff07fcfe9438916e3fcba7841894fbc07a WatchSource:0}: Error finding container 140952d1141cd425341b7f8b75e175ff07fcfe9438916e3fcba7841894fbc07a: Status 404 returned error can't find the container with id 140952d1141cd425341b7f8b75e175ff07fcfe9438916e3fcba7841894fbc07a Jul 06 22:46:28.522918 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:28.451979 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:46:29.056031 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.054102 2561 apiserver.go:52] "Watching apiserver" Jul 06 22:46:29.118623 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.081292 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 22:46:29.239348 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.219949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" event={"ID":"bfc2ad129405fbfb160c1db3475eb64b","Type":"ContainerStarted","Data":"140952d1141cd425341b7f8b75e175ff07fcfe9438916e3fcba7841894fbc07a"} Jul 06 22:46:29.239348 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.224447 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" event={"ID":"301ed78f06a8f545f44b18fb2b42ca58","Type":"ContainerStarted","Data":"eded05ba5ee28d80d2e958409a9c98cfa6c24a7e99193dea896b2f3e6ad2ea39"} Jul 06 22:46:29.252825 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.252795 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:46:29.256027 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.255894 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 22:46:29.256326 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.256294 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-wgbmm","openshift-image-registry/node-ca-qlk8d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal","openshift-multus/network-metrics-daemon-82jtz","openshift-network-diagnostics/network-check-target-c884g","kube-system/konnectivity-agent-l5jvj","openshift-dns/node-resolver-lgv7m","openshift-multus/multus-additional-cni-plugins-bw8fc","openshift-multus/multus-pdb5z","openshift-network-operator/iptables-alerter-vgng9","openshift-ovn-kubernetes/ovnkube-node-sprk8","kube-system/global-pull-secret-syncer-qsln2","kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2"] Jul 06 22:46:29.257953 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.257925 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:46:29.258105 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.258088 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tlz7z" Jul 06 22:46:29.258888 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.258856 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.261154 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.261135 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 22:46:29.261442 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.261328 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 22:46:29.261442 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.261393 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f7t4d\"" Jul 06 22:46:29.261442 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.261424 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.263665 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.263636 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.264442 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.264418 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 22:46:29.264442 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.264430 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 22:46:29.264595 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.264523 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2z7cj\"" Jul 06 22:46:29.264595 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.264426 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 22:46:29.264714 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.264652 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 22:46:29.264766 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.264715 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 22:46:29.264962 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.264841 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 22:46:29.265854 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.265801 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 22:46:29.266035 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.266016 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 22:46:29.266189 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.266173 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kth7n\"" Jul 06 22:46:29.266277 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.266238 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 22:46:29.267809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.267399 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:29.267809 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.267477 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:29.270400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.270181 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:29.270400 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.270252 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:29.272670 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.272396 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.275087 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.274649 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 22:46:29.275087 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.274942 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-86knk\"" Jul 06 22:46:29.275217 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.275110 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 22:46:29.275217 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.275120 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.277809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.277283 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 22:46:29.277809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.277451 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 22:46:29.277809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.277580 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 22:46:29.277809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.277602 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 22:46:29.277809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.277630 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pc5sr\"" Jul 06 22:46:29.278877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.278839 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.281255 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.280913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.282062 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.281743 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 22:46:29.282062 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.282004 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 22:46:29.282219 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.282210 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fkc5\"" Jul 06 22:46:29.283548 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.283521 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.284964 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.284640 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 22:46:29.284964 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.284669 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kjn6w\"" Jul 06 22:46:29.284964 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.284719 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 22:46:29.284964 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.284795 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 22:46:29.286460 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.286441 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.287370 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.287334 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:46:29.287973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.287954 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 22:46:29.288223 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.288204 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 22:46:29.289132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.289111 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:46:29.290793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.289357 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 22:46:29.290793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.289358 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7www5\"" Jul 06 22:46:29.290793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.289487 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t96cf\"" Jul 06 22:46:29.292832 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.292501 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.292832 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.292590 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293407 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-systemd-units\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293451 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4hr\" (UniqueName: \"kubernetes.io/projected/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-kube-api-access-bj4hr\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-hosts-file\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293496 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7282657b-60b7-4d5f-976c-306625667e03-cni-binary-copy\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-cni-multus\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293539 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-kubelet\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293569 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293589 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-sys-fs\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293610 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-socket-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjwrx\" (UniqueName: \"kubernetes.io/projected/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-kube-api-access-fjwrx\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293654 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-slash\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.293844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.293674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-tmp-dir\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-kubelet\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294096 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-node-log\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294120 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-cni-netd\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294144 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ckfn\" (UniqueName: \"kubernetes.io/projected/5422cceb-c1fe-4581-847a-59883489e159-kube-api-access-7ckfn\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294169 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-socket-dir-parent\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294214 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-cni-bin\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-os-release\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294261 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-systemd\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovn-node-metrics-cert\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294309 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-system-cni-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-cni-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294363 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-log-socket\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294395 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovnkube-config\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294418 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-etc-selinux\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.294446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294443 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adafc6d8-8f87-46e4-9e8a-e113a64326e6-iptables-alerter-script\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294466 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adafc6d8-8f87-46e4-9e8a-e113a64326e6-host-slash\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294493 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294540 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e237774-91e5-4bbc-8cdb-cbc48de6d8d5-agent-certs\") pod \"konnectivity-agent-l5jvj\" (UID: \"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5\") " pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cnibin\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294717 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294756 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-k8s-cni-cncf-io\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294792 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovnkube-script-lib\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294818 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-system-cni-dir\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294842 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3eb86e34-7ac4-49e6-910d-a04169181619-serviceca\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294869 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-etc-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294891 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-device-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294913 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4x2p\" (UniqueName: \"kubernetes.io/projected/35fc2ae6-58f8-476f-9a32-744af403179d-kube-api-access-k4x2p\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294935 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-hostroot\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.295228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.294957 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3eb86e34-7ac4-49e6-910d-a04169181619-host\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295005 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz829\" (UniqueName: \"kubernetes.io/projected/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-kube-api-access-kz829\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e237774-91e5-4bbc-8cdb-cbc48de6d8d5-konnectivity-ca\") pod \"konnectivity-agent-l5jvj\" (UID: \"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5\") " pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295068 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cni-binary-copy\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295108 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cf7z\" (UniqueName: \"kubernetes.io/projected/adafc6d8-8f87-46e4-9e8a-e113a64326e6-kube-api-access-6cf7z\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295131 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295157 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-netns\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295180 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2cf4\" (UniqueName: \"kubernetes.io/projected/7282657b-60b7-4d5f-976c-306625667e03-kube-api-access-c2cf4\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-ovn\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295234 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-cni-bin\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295255 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7282657b-60b7-4d5f-976c-306625667e03-multus-daemon-config\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295303 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-multus-certs\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295337 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-env-overrides\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295362 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295404 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295448 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-os-release\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295471 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-etc-kubernetes\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295524 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk4lt\" (UniqueName: \"kubernetes.io/projected/3eb86e34-7ac4-49e6-910d-a04169181619-kube-api-access-tk4lt\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.296877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-run-netns\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.296877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295595 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-var-lib-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.296877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295618 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-run-ovn-kubernetes\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.296877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-registration-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.296877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295708 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-cnibin\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.296877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.295733 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-conf-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.344150 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.344112 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:46:29.382622 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.382587 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 22:46:29.396316 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396261 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.396316 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396302 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-os-release\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396333 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-etc-kubernetes\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk4lt\" (UniqueName: \"kubernetes.io/projected/3eb86e34-7ac4-49e6-910d-a04169181619-kube-api-access-tk4lt\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396388 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysctl-d\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396415 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-sys\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396438 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-run-netns\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-var-lib-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396481 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-run-ovn-kubernetes\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396481 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396472 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-os-release\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.396535 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396506 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-registration-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396549 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-run-netns\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396560 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-etc-kubernetes\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396606 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-cnibin\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396608 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-run-ovn-kubernetes\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396633 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-conf-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396675 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-modprobe-d\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396720 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-lib-modules\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-systemd-units\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396773 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4hr\" (UniqueName: \"kubernetes.io/projected/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-kube-api-access-bj4hr\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-hosts-file\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7282657b-60b7-4d5f-976c-306625667e03-cni-binary-copy\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-cni-multus\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396892 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-var-lib-kubelet\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396919 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-kubelet\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396940 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-var-lib-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396975 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.396986 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-systemd-units\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397000 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-sys-fs\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397033 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-hosts-file\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397048 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-socket-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397069 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-cni-multus\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397077 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjwrx\" (UniqueName: \"kubernetes.io/projected/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-kube-api-access-fjwrx\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397125 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-cnibin\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397159 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-conf-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397203 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-kubernetes\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-kubelet\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397239 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-slash\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397264 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-registration-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397270 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-tmp-dir\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397323 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-slash\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397363 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-kubelet\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397400 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-systemd\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397446 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-run\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397478 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-tuned\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.397834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397503 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-node-log\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397530 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-cni-netd\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397556 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ckfn\" (UniqueName: \"kubernetes.io/projected/5422cceb-c1fe-4581-847a-59883489e159-kube-api-access-7ckfn\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-socket-dir-parent\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397610 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7282657b-60b7-4d5f-976c-306625667e03-cni-binary-copy\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397611 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-tmp-dir\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397656 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-node-log\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397704 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397716 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-cni-netd\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397738 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-kubelet\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397832 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-socket-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397879 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-sys-fs\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397882 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/436aaeeb-0306-4569-ad7b-25407a1c4caf-tmp\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397911 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-cni-bin\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397933 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-os-release\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397955 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397976 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvp4z\" (UniqueName: \"kubernetes.io/projected/436aaeeb-0306-4569-ad7b-25407a1c4caf-kube-api-access-nvp4z\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.398661 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.397998 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-cni-bin\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398008 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-socket-dir-parent\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398001 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-systemd\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398056 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-systemd\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398066 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovn-node-metrics-cert\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-system-cni-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398116 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-os-release\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-cni-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-kubelet-config\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398318 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-system-cni-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398322 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-multus-cni-dir\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-log-socket\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovnkube-config\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-etc-selinux\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398432 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adafc6d8-8f87-46e4-9e8a-e113a64326e6-iptables-alerter-script\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adafc6d8-8f87-46e4-9e8a-e113a64326e6-host-slash\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398479 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:29.399481 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398505 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398532 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e237774-91e5-4bbc-8cdb-cbc48de6d8d5-agent-certs\") pod \"konnectivity-agent-l5jvj\" (UID: \"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5\") " pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398555 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cnibin\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398582 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398591 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-k8s-cni-cncf-io\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysctl-conf\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398681 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovnkube-script-lib\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398720 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-system-cni-dir\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398740 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3eb86e34-7ac4-49e6-910d-a04169181619-serviceca\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398760 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-etc-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398781 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-device-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398805 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4x2p\" (UniqueName: \"kubernetes.io/projected/35fc2ae6-58f8-476f-9a32-744af403179d-kube-api-access-k4x2p\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-hostroot\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398867 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3eb86e34-7ac4-49e6-910d-a04169181619-host\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-dbus\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.400292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398915 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-host\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398939 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz829\" (UniqueName: \"kubernetes.io/projected/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-kube-api-access-kz829\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.398973 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e237774-91e5-4bbc-8cdb-cbc48de6d8d5-konnectivity-ca\") pod \"konnectivity-agent-l5jvj\" (UID: \"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5\") " pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399042 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adafc6d8-8f87-46e4-9e8a-e113a64326e6-iptables-alerter-script\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399058 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adafc6d8-8f87-46e4-9e8a-e113a64326e6-host-slash\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399057 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399104 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-k8s-cni-cncf-io\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399109 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cnibin\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399104 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-device-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399160 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-hostroot\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399199 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3eb86e34-7ac4-49e6-910d-a04169181619-host\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399229 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cni-binary-copy\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cf7z\" (UniqueName: \"kubernetes.io/projected/adafc6d8-8f87-46e4-9e8a-e113a64326e6-kube-api-access-6cf7z\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-netns\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399337 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2cf4\" (UniqueName: \"kubernetes.io/projected/7282657b-60b7-4d5f-976c-306625667e03-kube-api-access-c2cf4\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399367 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-ovn\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.401526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399392 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-cni-bin\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7282657b-60b7-4d5f-976c-306625667e03-multus-daemon-config\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399461 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-etc-selinux\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399490 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-system-cni-dir\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399546 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-multus-certs\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399546 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-var-lib-cni-bin\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399585 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-run-ovn\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399626 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysconfig\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399634 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399657 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-env-overrides\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399699 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35fc2ae6-58f8-476f-9a32-744af403179d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399817 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3eb86e34-7ac4-49e6-910d-a04169181619-serviceca\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.399966 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.400043 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7282657b-60b7-4d5f-976c-306625667e03-multus-daemon-config\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.400094 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-etc-openvswitch\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.400201 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-cni-binary-copy\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.400252 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-env-overrides\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.402327 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.400460 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:29.403166 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.400549 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs podName:5422cceb-c1fe-4581-847a-59883489e159 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:29.900529364 +0000 UTC m=+3.200230944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs") pod "network-metrics-daemon-82jtz" (UID: "5422cceb-c1fe-4581-847a-59883489e159") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:29.403166 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.400672 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6e237774-91e5-4bbc-8cdb-cbc48de6d8d5-konnectivity-ca\") pod \"konnectivity-agent-l5jvj\" (UID: \"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5\") " pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.403166 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.400719 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-netns\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.403166 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.400990 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-log-socket\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.403166 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.401036 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7282657b-60b7-4d5f-976c-306625667e03-host-run-multus-certs\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.403166 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.401263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovnkube-config\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.403166 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.401336 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovnkube-script-lib\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.404437 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.404413 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-ovn-node-metrics-cert\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.404580 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.404544 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6e237774-91e5-4bbc-8cdb-cbc48de6d8d5-agent-certs\") pod \"konnectivity-agent-l5jvj\" (UID: \"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5\") " pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.407202 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.407166 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjwrx\" (UniqueName: \"kubernetes.io/projected/4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3-kube-api-access-fjwrx\") pod \"node-resolver-lgv7m\" (UID: \"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3\") " pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.407589 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.407385 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk4lt\" (UniqueName: \"kubernetes.io/projected/3eb86e34-7ac4-49e6-910d-a04169181619-kube-api-access-tk4lt\") pod \"node-ca-qlk8d\" (UID: \"3eb86e34-7ac4-49e6-910d-a04169181619\") " pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.408552 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.408509 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ckfn\" (UniqueName: \"kubernetes.io/projected/5422cceb-c1fe-4581-847a-59883489e159-kube-api-access-7ckfn\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:29.408988 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.408964 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4x2p\" (UniqueName: \"kubernetes.io/projected/35fc2ae6-58f8-476f-9a32-744af403179d-kube-api-access-k4x2p\") pod \"aws-ebs-csi-driver-node-bg8p2\" (UID: \"35fc2ae6-58f8-476f-9a32-744af403179d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.410907 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.410536 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:46:29.410907 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.410560 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:46:29.410907 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.410575 2561 projected.go:196] Error preparing data for projected volume kube-api-access-cprnr for pod openshift-network-diagnostics/network-check-target-c884g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:29.410907 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.410633 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr podName:6e2aa88e-8b84-4997-acc8-162e6ce24a45 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:29.91061516 +0000 UTC m=+3.210316757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cprnr" (UniqueName: "kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr") pod "network-check-target-c884g" (UID: "6e2aa88e-8b84-4997-acc8-162e6ce24a45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:29.410907 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.410775 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4hr\" (UniqueName: \"kubernetes.io/projected/ad332d8f-fbd9-4a1d-8ff0-3473e9d50590-kube-api-access-bj4hr\") pod \"multus-additional-cni-plugins-bw8fc\" (UID: \"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590\") " pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.411720 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.411658 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz829\" (UniqueName: \"kubernetes.io/projected/720f5ab6-a3bb-4e67-8125-7d222c1d9b9d-kube-api-access-kz829\") pod \"ovnkube-node-sprk8\" (UID: \"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d\") " pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.411801 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.411730 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cf7z\" (UniqueName: \"kubernetes.io/projected/adafc6d8-8f87-46e4-9e8a-e113a64326e6-kube-api-access-6cf7z\") pod \"iptables-alerter-vgng9\" (UID: \"adafc6d8-8f87-46e4-9e8a-e113a64326e6\") " pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.413997 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.413975 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2cf4\" (UniqueName: \"kubernetes.io/projected/7282657b-60b7-4d5f-976c-306625667e03-kube-api-access-c2cf4\") pod \"multus-pdb5z\" (UID: \"7282657b-60b7-4d5f-976c-306625667e03\") " pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.500733 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500620 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-systemd\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.500733 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-run\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.500952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500769 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-run\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.500952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500771 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-systemd\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.500952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500805 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-tuned\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.500952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/436aaeeb-0306-4569-ad7b-25407a1c4caf-tmp\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.500952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500905 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.500952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvp4z\" (UniqueName: \"kubernetes.io/projected/436aaeeb-0306-4569-ad7b-25407a1c4caf-kube-api-access-nvp4z\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.500968 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-kubelet-config\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501017 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysctl-conf\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501048 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-dbus\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-host\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501111 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysconfig\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysctl-d\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501164 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-sys\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501192 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-modprobe-d\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501217 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-lib-modules\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-var-lib-kubelet\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-kubernetes\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501336 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-kubelet-config\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501374 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-kubernetes\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501426 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysconfig\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501479 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysctl-conf\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.501048 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501546 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-dbus\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501558 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-sysctl-d\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.501604 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret podName:fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:30.001581147 +0000 UTC m=+3.301282742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret") pod "global-pull-secret-syncer-qsln2" (UID: "fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501646 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-lib-modules\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.501657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501647 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-modprobe-d\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.502299 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501707 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-host\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.502299 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501718 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-sys\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.502299 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.501756 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/436aaeeb-0306-4569-ad7b-25407a1c4caf-var-lib-kubelet\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.503309 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.503289 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/436aaeeb-0306-4569-ad7b-25407a1c4caf-tmp\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.503501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.503481 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/436aaeeb-0306-4569-ad7b-25407a1c4caf-etc-tuned\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.509782 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.509752 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvp4z\" (UniqueName: \"kubernetes.io/projected/436aaeeb-0306-4569-ad7b-25407a1c4caf-kube-api-access-nvp4z\") pod \"tuned-wgbmm\" (UID: \"436aaeeb-0306-4569-ad7b-25407a1c4caf\") " pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.573718 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.573669 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lgv7m" Jul 06 22:46:29.582853 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.582815 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:29.595571 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.595538 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" Jul 06 22:46:29.601498 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.601472 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:29.610336 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.610312 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pdb5z" Jul 06 22:46:29.618159 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.618136 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" Jul 06 22:46:29.627835 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.627808 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qlk8d" Jul 06 22:46:29.635554 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.635533 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vgng9" Jul 06 22:46:29.642229 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.642206 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" Jul 06 22:46:29.903088 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:29.903008 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:29.903230 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.903134 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:29.903230 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:29.903197 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs podName:5422cceb-c1fe-4581-847a-59883489e159 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:30.903181119 +0000 UTC m=+4.202882909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs") pod "network-metrics-daemon-82jtz" (UID: "5422cceb-c1fe-4581-847a-59883489e159") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:30.003851 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:30.003813 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:30.004026 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:30.003924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:30.004026 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.003984 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:46:30.004026 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.004010 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:46:30.004026 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.004016 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:30.004026 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.004024 2561 projected.go:196] Error preparing data for projected volume kube-api-access-cprnr for pod openshift-network-diagnostics/network-check-target-c884g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:30.004239 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.004080 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret podName:fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:31.004065 +0000 UTC m=+4.303766598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret") pod "global-pull-secret-syncer-qsln2" (UID: "fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:30.004239 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.004096 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr podName:6e2aa88e-8b84-4997-acc8-162e6ce24a45 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:31.00409029 +0000 UTC m=+4.303791870 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cprnr" (UniqueName: "kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr") pod "network-check-target-c884g" (UID: "6e2aa88e-8b84-4997-acc8-162e6ce24a45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:30.263331 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:30.263213 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:41:29 +0000 UTC" deadline="2028-01-29 22:25:58.288288263 +0000 UTC" Jul 06 22:46:30.263331 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:30.263273 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13727h39m28.025019233s" Jul 06 22:46:30.454003 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.453963 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadafc6d8_8f87_46e4_9e8a_e113a64326e6.slice/crio-421c2a95bcf990b370dc7fecd307a622a3c80a551e2a8814dae7b212f29858de WatchSource:0}: Error finding container 421c2a95bcf990b370dc7fecd307a622a3c80a551e2a8814dae7b212f29858de: Status 404 returned error can't find the container with id 421c2a95bcf990b370dc7fecd307a622a3c80a551e2a8814dae7b212f29858de Jul 06 22:46:30.455872 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.455840 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35fc2ae6_58f8_476f_9a32_744af403179d.slice/crio-cce4d12ceb670073e80cb85382338c57ae734d12367e09aeeb81d831f09659a6 WatchSource:0}: Error finding container cce4d12ceb670073e80cb85382338c57ae734d12367e09aeeb81d831f09659a6: Status 404 returned error can't find the container with id cce4d12ceb670073e80cb85382338c57ae734d12367e09aeeb81d831f09659a6 Jul 06 22:46:30.459504 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.459472 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e237774_91e5_4bbc_8cdb_cbc48de6d8d5.slice/crio-20ea61697cba52925947f5e758bae7aa0eb3dc67ab2ebad88bb2c2c2aecf70da WatchSource:0}: Error finding container 20ea61697cba52925947f5e758bae7aa0eb3dc67ab2ebad88bb2c2c2aecf70da: Status 404 returned error can't find the container with id 20ea61697cba52925947f5e758bae7aa0eb3dc67ab2ebad88bb2c2c2aecf70da Jul 06 22:46:30.460116 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.459994 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb86e34_7ac4_49e6_910d_a04169181619.slice/crio-1d82c534b239f7809a6e05e725df553eb8afe7715db85d9558bc952fc3801f65 WatchSource:0}: Error finding container 1d82c534b239f7809a6e05e725df553eb8afe7715db85d9558bc952fc3801f65: Status 404 returned error can't find the container with id 1d82c534b239f7809a6e05e725df553eb8afe7715db85d9558bc952fc3801f65 Jul 06 22:46:30.461150 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.461127 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720f5ab6_a3bb_4e67_8125_7d222c1d9b9d.slice/crio-c74a53bd0a8ab3f97075ef1f7949e4d15cd087e94e5c7495a69ea6232f37e7a1 WatchSource:0}: Error finding container c74a53bd0a8ab3f97075ef1f7949e4d15cd087e94e5c7495a69ea6232f37e7a1: Status 404 returned error can't find the container with id c74a53bd0a8ab3f97075ef1f7949e4d15cd087e94e5c7495a69ea6232f37e7a1 Jul 06 22:46:30.462141 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.462117 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7282657b_60b7_4d5f_976c_306625667e03.slice/crio-03c03ef8c25fc59016bfafc018514641bbea460d41f4b0e4b3a3c14ee18b35d5 WatchSource:0}: Error finding container 03c03ef8c25fc59016bfafc018514641bbea460d41f4b0e4b3a3c14ee18b35d5: Status 404 returned error can't find the container with id 03c03ef8c25fc59016bfafc018514641bbea460d41f4b0e4b3a3c14ee18b35d5 Jul 06 22:46:30.463731 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.463665 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad332d8f_fbd9_4a1d_8ff0_3473e9d50590.slice/crio-c6ae2ae88045f670c10455d1103e8b669d51e932b11ce46bd9a1fb7f1ef0473b WatchSource:0}: Error finding container c6ae2ae88045f670c10455d1103e8b669d51e932b11ce46bd9a1fb7f1ef0473b: Status 404 returned error can't find the container with id c6ae2ae88045f670c10455d1103e8b669d51e932b11ce46bd9a1fb7f1ef0473b Jul 06 22:46:30.464524 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.464500 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436aaeeb_0306_4569_ad7b_25407a1c4caf.slice/crio-f3d4b984e58985066156d56e29ba02e48d7dd90df482db87b27e34697563bd64 WatchSource:0}: Error finding container f3d4b984e58985066156d56e29ba02e48d7dd90df482db87b27e34697563bd64: Status 404 returned error can't find the container with id f3d4b984e58985066156d56e29ba02e48d7dd90df482db87b27e34697563bd64 Jul 06 22:46:30.466209 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:46:30.466178 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4027f84a_d2a8_4d7e_bdcb_3c7ab7642bc3.slice/crio-f4c44c7ce495313e4faa9f244a8a2dd26d0cfcc135589d8763da23ebfbf9eb6b WatchSource:0}: Error finding container f4c44c7ce495313e4faa9f244a8a2dd26d0cfcc135589d8763da23ebfbf9eb6b: Status 404 returned error can't find the container with id f4c44c7ce495313e4faa9f244a8a2dd26d0cfcc135589d8763da23ebfbf9eb6b Jul 06 22:46:30.910953 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:30.910843 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:30.911120 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.911037 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:30.911120 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:30.911099 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs podName:5422cceb-c1fe-4581-847a-59883489e159 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:32.911082509 +0000 UTC m=+6.210784102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs") pod "network-metrics-daemon-82jtz" (UID: "5422cceb-c1fe-4581-847a-59883489e159") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:31.011473 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.011404 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:31.011473 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.011463 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:31.011726 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.011649 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:46:31.011726 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.011671 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:46:31.011726 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.011701 2561 projected.go:196] Error preparing data for projected volume kube-api-access-cprnr for pod openshift-network-diagnostics/network-check-target-c884g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:31.011866 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.011765 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr podName:6e2aa88e-8b84-4997-acc8-162e6ce24a45 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:33.011745292 +0000 UTC m=+6.311446885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cprnr" (UniqueName: "kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr") pod "network-check-target-c884g" (UID: "6e2aa88e-8b84-4997-acc8-162e6ce24a45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:31.012194 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.012176 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:31.012261 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.012234 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret podName:fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:33.012218571 +0000 UTC m=+6.311920157 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret") pod "global-pull-secret-syncer-qsln2" (UID: "fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:31.214900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.214813 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:31.214900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.214813 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:31.214900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.214869 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:31.215161 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.214993 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:31.215622 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.215435 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:31.215622 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:31.215523 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:31.232513 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.232479 2561 generic.go:356] "Generic (PLEG): container finished" podID="bfc2ad129405fbfb160c1db3475eb64b" containerID="b281c6898f192e021f6887aeab181dabfdfc3650f8f12610118fc5bb00cc04b0" exitCode=0 Jul 06 22:46:31.232649 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.232567 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" event={"ID":"bfc2ad129405fbfb160c1db3475eb64b","Type":"ContainerDied","Data":"b281c6898f192e021f6887aeab181dabfdfc3650f8f12610118fc5bb00cc04b0"} Jul 06 22:46:31.257227 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.257192 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" event={"ID":"301ed78f06a8f545f44b18fb2b42ca58","Type":"ContainerStarted","Data":"43b7272735f6985f263fddacad7f82204801efcb05f4a2764997fb94e0442151"} Jul 06 22:46:31.264145 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.264067 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:41:29 +0000 UTC" deadline="2028-04-04 14:57:54.275822246 +0000 UTC" Jul 06 22:46:31.264145 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.264119 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15304h11m23.011708132s" Jul 06 22:46:31.266434 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.266339 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lgv7m" event={"ID":"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3","Type":"ContainerStarted","Data":"f4c44c7ce495313e4faa9f244a8a2dd26d0cfcc135589d8763da23ebfbf9eb6b"} Jul 06 22:46:31.273786 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.273722 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pdb5z" event={"ID":"7282657b-60b7-4d5f-976c-306625667e03","Type":"ContainerStarted","Data":"03c03ef8c25fc59016bfafc018514641bbea460d41f4b0e4b3a3c14ee18b35d5"} Jul 06 22:46:31.297447 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.297408 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qlk8d" event={"ID":"3eb86e34-7ac4-49e6-910d-a04169181619","Type":"ContainerStarted","Data":"1d82c534b239f7809a6e05e725df553eb8afe7715db85d9558bc952fc3801f65"} Jul 06 22:46:31.317270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.317182 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5jvj" event={"ID":"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5","Type":"ContainerStarted","Data":"20ea61697cba52925947f5e758bae7aa0eb3dc67ab2ebad88bb2c2c2aecf70da"} Jul 06 22:46:31.328963 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.328905 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vgng9" event={"ID":"adafc6d8-8f87-46e4-9e8a-e113a64326e6","Type":"ContainerStarted","Data":"421c2a95bcf990b370dc7fecd307a622a3c80a551e2a8814dae7b212f29858de"} Jul 06 22:46:31.342655 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.342613 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" event={"ID":"436aaeeb-0306-4569-ad7b-25407a1c4caf","Type":"ContainerStarted","Data":"f3d4b984e58985066156d56e29ba02e48d7dd90df482db87b27e34697563bd64"} Jul 06 22:46:31.358598 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.358559 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerStarted","Data":"c6ae2ae88045f670c10455d1103e8b669d51e932b11ce46bd9a1fb7f1ef0473b"} Jul 06 22:46:31.372438 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.372311 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"c74a53bd0a8ab3f97075ef1f7949e4d15cd087e94e5c7495a69ea6232f37e7a1"} Jul 06 22:46:31.390884 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:31.390844 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" event={"ID":"35fc2ae6-58f8-476f-9a32-744af403179d","Type":"ContainerStarted","Data":"cce4d12ceb670073e80cb85382338c57ae734d12367e09aeeb81d831f09659a6"} Jul 06 22:46:32.409354 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:32.409256 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" event={"ID":"bfc2ad129405fbfb160c1db3475eb64b","Type":"ContainerStarted","Data":"65d667e4f373f08ee9aeb9308c4f18d0f0a47b7396232802e7cb995820e08520"} Jul 06 22:46:32.423940 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:32.423877 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" podStartSLOduration=4.42385208 podStartE2EDuration="4.42385208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:31.272224996 +0000 UTC m=+4.571926616" watchObservedRunningTime="2026-07-06 22:46:32.42385208 +0000 UTC m=+5.723553685" Jul 06 22:46:32.930149 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:32.930108 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:32.930344 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:32.930277 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:32.930344 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:32.930340 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs podName:5422cceb-c1fe-4581-847a-59883489e159 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:36.930321624 +0000 UTC m=+10.230023209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs") pod "network-metrics-daemon-82jtz" (UID: "5422cceb-c1fe-4581-847a-59883489e159") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:33.031924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:33.031979 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.032141 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.032159 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.032175 2561 projected.go:196] Error preparing data for projected volume kube-api-access-cprnr for pod openshift-network-diagnostics/network-check-target-c884g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.032236 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr podName:6e2aa88e-8b84-4997-acc8-162e6ce24a45 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:37.032216601 +0000 UTC m=+10.331918187 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cprnr" (UniqueName: "kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr") pod "network-check-target-c884g" (UID: "6e2aa88e-8b84-4997-acc8-162e6ce24a45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.032614 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:33.032752 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.032675 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret podName:fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:37.032649386 +0000 UTC m=+10.332350989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret") pod "global-pull-secret-syncer-qsln2" (UID: "fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:33.216553 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:33.216472 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:33.216726 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.216602 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:33.216996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:33.216978 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:33.217107 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.217089 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:33.217163 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:33.217136 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:33.217230 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:33.217212 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:35.214711 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:35.213927 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:35.214711 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:35.213975 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:35.214711 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:35.213927 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:35.214711 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:35.214067 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:35.214711 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:35.214212 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:35.214711 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:35.214293 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:36.967806 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:36.967772 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:36.968286 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:36.967962 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:36.968286 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:36.968038 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs podName:5422cceb-c1fe-4581-847a-59883489e159 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:44.968016807 +0000 UTC m=+18.267718402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs") pod "network-metrics-daemon-82jtz" (UID: "5422cceb-c1fe-4581-847a-59883489e159") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:37.068489 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:37.068448 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:37.068673 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:37.068509 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:37.068673 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.068627 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:37.068819 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.068675 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:46:37.068819 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.068714 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:46:37.068819 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.068726 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret podName:fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:45.068704389 +0000 UTC m=+18.368406012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret") pod "global-pull-secret-syncer-qsln2" (UID: "fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:37.068819 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.068728 2561 projected.go:196] Error preparing data for projected volume kube-api-access-cprnr for pod openshift-network-diagnostics/network-check-target-c884g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:37.068819 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.068780 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr podName:6e2aa88e-8b84-4997-acc8-162e6ce24a45 nodeName:}" failed. No retries permitted until 2026-07-06 22:46:45.068769702 +0000 UTC m=+18.368471295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cprnr" (UniqueName: "kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr") pod "network-check-target-c884g" (UID: "6e2aa88e-8b84-4997-acc8-162e6ce24a45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:37.215606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:37.215572 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:37.215775 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.215702 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:37.216117 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:37.216096 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:37.216249 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.216215 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:37.216465 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:37.216346 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:37.216465 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:37.216433 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:39.214032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:39.213987 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:39.214506 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:39.214123 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:39.214881 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:39.214572 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:39.214881 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:39.214701 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:39.214881 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:39.214771 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:39.214881 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:39.214840 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:41.214457 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:41.214420 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:41.214915 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:41.214420 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:41.214915 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:41.214559 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:41.214915 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:41.214605 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:41.214915 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:41.214422 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:41.214915 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:41.214701 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:43.213834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:43.213798 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:43.213834 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:43.213816 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:43.214403 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:43.213933 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:43.214403 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:43.213941 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:43.214403 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:43.213997 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:43.214403 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:43.214092 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:45.027419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:45.027376 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:45.027882 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.027526 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:45.027882 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.027592 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs podName:5422cceb-c1fe-4581-847a-59883489e159 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:01.02757069 +0000 UTC m=+34.327272287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs") pod "network-metrics-daemon-82jtz" (UID: "5422cceb-c1fe-4581-847a-59883489e159") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:46:45.128231 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:45.128204 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:45.128314 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:45.128241 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:45.128376 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.128363 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:46:45.128413 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.128381 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:46:45.128413 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.128391 2561 projected.go:196] Error preparing data for projected volume kube-api-access-cprnr for pod openshift-network-diagnostics/network-check-target-c884g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:45.128514 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.128433 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr podName:6e2aa88e-8b84-4997-acc8-162e6ce24a45 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:01.12842163 +0000 UTC m=+34.428123209 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cprnr" (UniqueName: "kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr") pod "network-check-target-c884g" (UID: "6e2aa88e-8b84-4997-acc8-162e6ce24a45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:46:45.128514 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.128364 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:45.128593 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.128516 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret podName:fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:01.128494464 +0000 UTC m=+34.428196066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret") pod "global-pull-secret-syncer-qsln2" (UID: "fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:46:45.214700 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:45.214597 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:45.214700 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:45.214638 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:45.214925 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.214756 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:45.214925 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.214803 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:45.214925 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:45.214875 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:45.215089 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:45.215000 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:47.214975 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:47.214935 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:47.215410 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:47.215039 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:47.215410 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:47.215089 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:47.215410 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:47.215199 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:47.215410 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:47.215224 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:47.215410 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:47.215338 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:49.214510 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:49.214470 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:49.215066 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:49.214470 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:49.215066 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:49.214622 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:49.215066 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:49.214470 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:49.215066 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:49.214765 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:49.215066 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:49.214869 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:50.450102 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.449889 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" event={"ID":"35fc2ae6-58f8-476f-9a32-744af403179d","Type":"ContainerStarted","Data":"6579d811508d9d966a507449b40c7205301918a0bc0cf47c0b742c0cc30bbd10"} Jul 06 22:46:50.451278 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.451250 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lgv7m" event={"ID":"4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3","Type":"ContainerStarted","Data":"7cec5e9eb219611fd72e9ea2dae80deeaa9ccda5cc9d1a8815143a9d8c6b5d36"} Jul 06 22:46:50.452769 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.452746 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pdb5z" event={"ID":"7282657b-60b7-4d5f-976c-306625667e03","Type":"ContainerStarted","Data":"35025c794adec4685fb340179595eddd862515d952014b4d9e285aebd52a1bee"} Jul 06 22:46:50.454162 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.454143 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qlk8d" event={"ID":"3eb86e34-7ac4-49e6-910d-a04169181619","Type":"ContainerStarted","Data":"f94e7ad2690be2e60db385666ed40ff1413ea63a30856190967590c4180cb82f"} Jul 06 22:46:50.455511 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.455483 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l5jvj" event={"ID":"6e237774-91e5-4bbc-8cdb-cbc48de6d8d5","Type":"ContainerStarted","Data":"a470d41d5e065dbe1f983171e13e2c498886b2feebd228005e52593c0d7f2387"} Jul 06 22:46:50.456809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.456790 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" event={"ID":"436aaeeb-0306-4569-ad7b-25407a1c4caf","Type":"ContainerStarted","Data":"a3703d6cddb2c6e90b34667e0d4bc39e52ebaca071277f60727d200e73926d11"} Jul 06 22:46:50.458158 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.458130 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad332d8f-fbd9-4a1d-8ff0-3473e9d50590" containerID="8b854fe5503d3fe367b3997c62cca5e7eb232a845b24a762efada378979e9fd1" exitCode=0 Jul 06 22:46:50.458231 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.458201 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerDied","Data":"8b854fe5503d3fe367b3997c62cca5e7eb232a845b24a762efada378979e9fd1"} Jul 06 22:46:50.460923 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.460907 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:46:50.461251 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.461232 2561 generic.go:356] "Generic (PLEG): container finished" podID="720f5ab6-a3bb-4e67-8125-7d222c1d9b9d" containerID="676790d85ec457b780150ddde012ee0f942897e3b918878e6441926e664dcfed" exitCode=1 Jul 06 22:46:50.461331 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.461266 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"e43cf9fe5631375ed0d9d71f20a5af7de4820f6f62bf749b0c588f87b31ab553"} Jul 06 22:46:50.461401 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.461333 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"9cce52fba67c41c995cb7c21eacb72e4d372e98ee3abb99e16ec4ee16aff3947"} Jul 06 22:46:50.461401 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.461390 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"a4a60b697de66d1e7beebc5a2619185444f8fb0e50ec1dd83774d6328d932c45"} Jul 06 22:46:50.461503 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.461439 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"8fe1e4bff0b261b3dafe7cbec86045eba05f3898a342c26913ef31a29555cda8"} Jul 06 22:46:50.461503 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.461486 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerDied","Data":"676790d85ec457b780150ddde012ee0f942897e3b918878e6441926e664dcfed"} Jul 06 22:46:50.461600 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.461537 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"46d1a2345963a46ef710b52d2b83fb8291db2c8e13f488373f795377736767a7"} Jul 06 22:46:50.465637 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.465599 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" podStartSLOduration=22.465586132 podStartE2EDuration="22.465586132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:32.425119606 +0000 UTC m=+5.724821227" watchObservedRunningTime="2026-07-06 22:46:50.465586132 +0000 UTC m=+23.765287734" Jul 06 22:46:50.465805 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.465785 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lgv7m" podStartSLOduration=4.578928295 podStartE2EDuration="23.465779671s" totalImagesPullingTime="18.886851376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:50.465326497 +0000 UTC m=+23.765028361" watchObservedRunningTime="2026-07-06 22:46:50.465779671 +0000 UTC m=+23.765481272" Jul 06 22:46:50.480910 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.480848 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pdb5z" podStartSLOduration=4.570017683 podStartE2EDuration="23.480827561s" totalImagesPullingTime="18.910809878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:50.480261445 +0000 UTC m=+23.779963048" watchObservedRunningTime="2026-07-06 22:46:50.480827561 +0000 UTC m=+23.780529161" Jul 06 22:46:50.516628 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.516564 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wgbmm" podStartSLOduration=4.633665264 podStartE2EDuration="23.516549906s" totalImagesPullingTime="18.882884642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:50.516057538 +0000 UTC m=+23.815759144" watchObservedRunningTime="2026-07-06 22:46:50.516549906 +0000 UTC m=+23.816251508" Jul 06 22:46:50.529174 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.529121 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l5jvj" podStartSLOduration=4.672751978 podStartE2EDuration="23.529107945s" totalImagesPullingTime="18.856355967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:50.528842791 +0000 UTC m=+23.828544394" watchObservedRunningTime="2026-07-06 22:46:50.529107945 +0000 UTC m=+23.828809548" Jul 06 22:46:50.531091 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.531071 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:50.549109 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.549060 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qlk8d" podStartSLOduration=4.662741437 podStartE2EDuration="23.549043464s" totalImagesPullingTime="18.886302027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:50.548714814 +0000 UTC m=+23.848416417" watchObservedRunningTime="2026-07-06 22:46:50.549043464 +0000 UTC m=+23.848745066" Jul 06 22:46:50.844273 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:50.844241 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 22:46:51.199039 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.198920 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T22:46:50.844266783Z","UUID":"c59c0371-7c3e-40c5-95c9-af5f51648753","Handler":null,"Name":"","Endpoint":""} Jul 06 22:46:51.200610 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.200590 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 22:46:51.200775 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.200632 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 22:46:51.214135 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.214108 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:51.214310 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.214118 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:51.214310 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.214242 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:51.214310 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:51.214235 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:51.214452 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:51.214315 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:51.214452 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:51.214378 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:51.466125 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.466034 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vgng9" event={"ID":"adafc6d8-8f87-46e4-9e8a-e113a64326e6","Type":"ContainerStarted","Data":"745411e213cca9961c304198f1dc61a9b494ffc817189d2d4f7d61b591eecb27"} Jul 06 22:46:51.469142 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.469111 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" event={"ID":"35fc2ae6-58f8-476f-9a32-744af403179d","Type":"ContainerStarted","Data":"79683334c0bd4a93214764c715e2c95baf02679e03780694801db3ee81c39e3d"} Jul 06 22:46:51.482558 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:51.482498 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vgng9" podStartSLOduration=5.589405714 podStartE2EDuration="24.482477008s" totalImagesPullingTime="18.893071294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:51.481578742 +0000 UTC m=+24.781280344" watchObservedRunningTime="2026-07-06 22:46:51.482477008 +0000 UTC m=+24.782178614" Jul 06 22:46:52.474523 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:52.474236 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" event={"ID":"35fc2ae6-58f8-476f-9a32-744af403179d","Type":"ContainerStarted","Data":"11a106d6dcb8c5ad3266baf55bfdad6cedb8bedbaa721673c786d6e4d36dba60"} Jul 06 22:46:52.477806 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:52.477785 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:46:52.478245 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:52.478196 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"e0b401283c931bd91f98d0fbc56eea73f2987b806b7ac391459ec45b57e23ad6"} Jul 06 22:46:52.491975 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:52.491924 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg8p2" podStartSLOduration=4.184538304 podStartE2EDuration="25.491905009s" totalImagesPullingTime="21.307366705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:52.490439795 +0000 UTC m=+25.790141427" watchObservedRunningTime="2026-07-06 22:46:52.491905009 +0000 UTC m=+25.791606612" Jul 06 22:46:53.214611 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:53.214575 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:53.214824 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:53.214575 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:53.214824 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:53.214737 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:53.214824 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:53.214574 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:53.214824 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:53.214775 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:53.215045 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:53.214853 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:53.226778 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:53.226749 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:53.227367 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:53.227351 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:53.482390 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:53.482307 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l5jvj" Jul 06 22:46:55.214233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.214051 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:55.214814 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.214054 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:55.214814 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:55.214314 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:55.214814 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.214054 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:55.214814 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:55.214376 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:55.214814 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:55.214475 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:55.486289 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.486210 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad332d8f-fbd9-4a1d-8ff0-3473e9d50590" containerID="d1b8c0c1c34265f4100d383317ee0ec7e698c7b82a196a10ff843f37e1637095" exitCode=0 Jul 06 22:46:55.486469 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.486289 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerDied","Data":"d1b8c0c1c34265f4100d383317ee0ec7e698c7b82a196a10ff843f37e1637095"} Jul 06 22:46:55.489918 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.489900 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:46:55.490439 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.490422 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"3c40a564b7cbb83dbd58be64ae4d38e758ae5b274a0f1f72d04444347b0aa845"} Jul 06 22:46:55.490833 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:55.490817 2561 scope.go:117] "RemoveContainer" containerID="676790d85ec457b780150ddde012ee0f942897e3b918878e6441926e664dcfed" Jul 06 22:46:56.464629 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.464590 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qsln2"] Jul 06 22:46:56.465067 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.464778 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:56.465067 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:56.464891 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:56.467751 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.467722 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c884g"] Jul 06 22:46:56.467866 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.467812 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:56.467931 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:56.467907 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:56.468373 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.468353 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-82jtz"] Jul 06 22:46:56.468467 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.468450 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:56.468557 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:56.468540 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:56.496947 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.496918 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:46:56.497312 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.497287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" event={"ID":"720f5ab6-a3bb-4e67-8125-7d222c1d9b9d","Type":"ContainerStarted","Data":"59bc86c56136e91fe704ac083708f8ff025b69a96c6accf6517da95fea5a4590"} Jul 06 22:46:56.497410 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.497398 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 22:46:56.497592 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.497569 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:56.497712 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.497636 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:56.514147 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.513955 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:56.514268 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.514207 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:56.527033 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:56.526988 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" podStartSLOduration=10.488004154 podStartE2EDuration="29.526957632s" totalImagesPullingTime="19.038953478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:46:56.525762111 +0000 UTC m=+29.825463732" watchObservedRunningTime="2026-07-06 22:46:56.526957632 +0000 UTC m=+29.826659234" Jul 06 22:46:57.504019 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:57.503979 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad332d8f-fbd9-4a1d-8ff0-3473e9d50590" containerID="08cdda167fdf3432076437247e29629767cc4198769250efbec34fbe0571da2f" exitCode=0 Jul 06 22:46:57.504586 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:57.504072 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerDied","Data":"08cdda167fdf3432076437247e29629767cc4198769250efbec34fbe0571da2f"} Jul 06 22:46:57.504586 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:57.504166 2561 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 22:46:57.881781 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:57.881709 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:46:58.214531 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:58.214494 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:46:58.214727 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:58.214539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:46:58.214727 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:58.214542 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:46:58.214727 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:58.214618 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:46:58.214727 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:58.214717 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:46:58.214944 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:46:58.214785 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:46:58.510028 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:58.509993 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerStarted","Data":"b9a3e5be4bb43d8b7a64053cd4ffef1a5ead367989c749deacf1a71dd6a5dd92"} Jul 06 22:46:59.514931 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:59.514843 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad332d8f-fbd9-4a1d-8ff0-3473e9d50590" containerID="b9a3e5be4bb43d8b7a64053cd4ffef1a5ead367989c749deacf1a71dd6a5dd92" exitCode=0 Jul 06 22:46:59.515533 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:59.514926 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerDied","Data":"b9a3e5be4bb43d8b7a64053cd4ffef1a5ead367989c749deacf1a71dd6a5dd92"} Jul 06 22:46:59.527235 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:46:59.527189 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" podUID="720f5ab6-a3bb-4e67-8125-7d222c1d9b9d" containerName="ovnkube-controller" probeResult="failure" output="" Jul 06 22:47:00.214463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:00.214423 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:00.214650 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:00.214491 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:47:00.214650 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:00.214494 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:47:00.214650 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:00.214578 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:47:00.214840 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:00.214670 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:47:00.214840 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:00.214770 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:47:01.049574 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:01.049351 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:47:01.050043 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.049528 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:47:01.050043 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.049740 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs podName:5422cceb-c1fe-4581-847a-59883489e159 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:33.0497196 +0000 UTC m=+66.349421189 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs") pod "network-metrics-daemon-82jtz" (UID: "5422cceb-c1fe-4581-847a-59883489e159") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:47:01.149968 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:01.149926 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:47:01.150151 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:01.149981 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:01.150151 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.150054 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:47:01.150151 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.150095 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:47:01.150151 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.150118 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:47:01.150151 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.150119 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret podName:fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:33.150100991 +0000 UTC m=+66.449802575 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret") pod "global-pull-secret-syncer-qsln2" (UID: "fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:47:01.150151 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.150130 2561 projected.go:196] Error preparing data for projected volume kube-api-access-cprnr for pod openshift-network-diagnostics/network-check-target-c884g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:47:01.150478 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:01.150181 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr podName:6e2aa88e-8b84-4997-acc8-162e6ce24a45 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:33.150165442 +0000 UTC m=+66.449867039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cprnr" (UniqueName: "kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr") pod "network-check-target-c884g" (UID: "6e2aa88e-8b84-4997-acc8-162e6ce24a45") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:47:02.213855 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:02.213822 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:47:02.214391 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:02.213926 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:47:02.214391 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:02.213944 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qsln2" podUID="fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44" Jul 06 22:47:02.214391 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:02.214006 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-82jtz" podUID="5422cceb-c1fe-4581-847a-59883489e159" Jul 06 22:47:02.214391 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:02.214031 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:02.214391 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:02.214154 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c884g" podUID="6e2aa88e-8b84-4997-acc8-162e6ce24a45" Jul 06 22:47:03.014839 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.014810 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeReady" Jul 06 22:47:03.015025 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.014956 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 22:47:03.054900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.054863 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8p8t2"] Jul 06 22:47:03.057745 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.057719 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.058206 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.057884 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wtsvg"] Jul 06 22:47:03.059753 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.059732 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.060400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.060367 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 22:47:03.060757 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.060705 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 22:47:03.060973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.060955 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ctk6j\"" Jul 06 22:47:03.063660 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.063599 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 22:47:03.063810 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.063725 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4v9t\"" Jul 06 22:47:03.063810 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.063726 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 22:47:03.064785 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.064745 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 22:47:03.070629 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.070272 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8p8t2"] Jul 06 22:47:03.070786 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.070763 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wtsvg"] Jul 06 22:47:03.162945 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.162861 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efb1a3a3-9752-443b-834d-6ae1b3c0b448-cert\") pod \"ingress-canary-wtsvg\" (UID: \"efb1a3a3-9752-443b-834d-6ae1b3c0b448\") " pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.162945 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.162897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d3599fe-dbea-4bdc-b913-846b419c5fac-tmp-dir\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.162945 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.162918 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3599fe-dbea-4bdc-b913-846b419c5fac-config-volume\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.163211 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.162994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.163211 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.163030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzc7v\" (UniqueName: \"kubernetes.io/projected/4d3599fe-dbea-4bdc-b913-846b419c5fac-kube-api-access-kzc7v\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.163211 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.163120 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb2lj\" (UniqueName: \"kubernetes.io/projected/efb1a3a3-9752-443b-834d-6ae1b3c0b448-kube-api-access-zb2lj\") pod \"ingress-canary-wtsvg\" (UID: \"efb1a3a3-9752-443b-834d-6ae1b3c0b448\") " pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.263920 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.263882 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efb1a3a3-9752-443b-834d-6ae1b3c0b448-cert\") pod \"ingress-canary-wtsvg\" (UID: \"efb1a3a3-9752-443b-834d-6ae1b3c0b448\") " pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.263930 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d3599fe-dbea-4bdc-b913-846b419c5fac-tmp-dir\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.263966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3599fe-dbea-4bdc-b913-846b419c5fac-config-volume\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.264002 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.264027 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzc7v\" (UniqueName: \"kubernetes.io/projected/4d3599fe-dbea-4bdc-b913-846b419c5fac-kube-api-access-kzc7v\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.264055 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb2lj\" (UniqueName: \"kubernetes.io/projected/efb1a3a3-9752-443b-834d-6ae1b3c0b448-kube-api-access-zb2lj\") pod \"ingress-canary-wtsvg\" (UID: \"efb1a3a3-9752-443b-834d-6ae1b3c0b448\") " pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.264365 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d3599fe-dbea-4bdc-b913-846b419c5fac-tmp-dir\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:03.264449 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:47:03.264501 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:03.264502 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls podName:4d3599fe-dbea-4bdc-b913-846b419c5fac nodeName:}" failed. No retries permitted until 2026-07-06 22:47:03.764488819 +0000 UTC m=+37.064190398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls") pod "dns-default-8p8t2" (UID: "4d3599fe-dbea-4bdc-b913-846b419c5fac") : secret "dns-default-metrics-tls" not found Jul 06 22:47:03.264984 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.264577 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d3599fe-dbea-4bdc-b913-846b419c5fac-config-volume\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.269138 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.269115 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efb1a3a3-9752-443b-834d-6ae1b3c0b448-cert\") pod \"ingress-canary-wtsvg\" (UID: \"efb1a3a3-9752-443b-834d-6ae1b3c0b448\") " pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.272437 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.272405 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb2lj\" (UniqueName: \"kubernetes.io/projected/efb1a3a3-9752-443b-834d-6ae1b3c0b448-kube-api-access-zb2lj\") pod \"ingress-canary-wtsvg\" (UID: \"efb1a3a3-9752-443b-834d-6ae1b3c0b448\") " pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.272545 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.272474 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzc7v\" (UniqueName: \"kubernetes.io/projected/4d3599fe-dbea-4bdc-b913-846b419c5fac-kube-api-access-kzc7v\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.378398 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.378371 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wtsvg" Jul 06 22:47:03.528671 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.528390 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerStarted","Data":"14242840bc98722009fad024fb412047b9e2a68cf0885553d3d678a966f8bf6f"} Jul 06 22:47:03.529866 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.529836 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wtsvg"] Jul 06 22:47:03.534249 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:03.534218 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb1a3a3_9752_443b_834d_6ae1b3c0b448.slice/crio-cb5d992006e0564d40c8db41b48ea32c26bb1da936ec2f0e1a51057a0c1fc6c1 WatchSource:0}: Error finding container cb5d992006e0564d40c8db41b48ea32c26bb1da936ec2f0e1a51057a0c1fc6c1: Status 404 returned error can't find the container with id cb5d992006e0564d40c8db41b48ea32c26bb1da936ec2f0e1a51057a0c1fc6c1 Jul 06 22:47:03.767392 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:03.767314 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:03.767523 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:03.767431 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:47:03.767523 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:03.767493 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls podName:4d3599fe-dbea-4bdc-b913-846b419c5fac nodeName:}" failed. No retries permitted until 2026-07-06 22:47:04.7674792 +0000 UTC m=+38.067180779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls") pod "dns-default-8p8t2" (UID: "4d3599fe-dbea-4bdc-b913-846b419c5fac") : secret "dns-default-metrics-tls" not found Jul 06 22:47:04.214479 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.214431 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:47:04.214479 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.214484 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:04.214762 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.214431 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:47:04.217082 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.217031 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:47:04.218113 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.218092 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:47:04.218230 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.218124 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 22:47:04.218230 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.218146 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmht5\"" Jul 06 22:47:04.218230 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.218092 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j9ddx\"" Jul 06 22:47:04.218230 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.218124 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 22:47:04.533144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.533055 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wtsvg" event={"ID":"efb1a3a3-9752-443b-834d-6ae1b3c0b448","Type":"ContainerStarted","Data":"cb5d992006e0564d40c8db41b48ea32c26bb1da936ec2f0e1a51057a0c1fc6c1"} Jul 06 22:47:04.536310 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.536275 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad332d8f-fbd9-4a1d-8ff0-3473e9d50590" containerID="14242840bc98722009fad024fb412047b9e2a68cf0885553d3d678a966f8bf6f" exitCode=0 Jul 06 22:47:04.536443 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.536350 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerDied","Data":"14242840bc98722009fad024fb412047b9e2a68cf0885553d3d678a966f8bf6f"} Jul 06 22:47:04.776551 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:04.776508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:04.776759 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:04.776678 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:47:04.776838 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:04.776791 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls podName:4d3599fe-dbea-4bdc-b913-846b419c5fac nodeName:}" failed. No retries permitted until 2026-07-06 22:47:06.776769984 +0000 UTC m=+40.076471565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls") pod "dns-default-8p8t2" (UID: "4d3599fe-dbea-4bdc-b913-846b419c5fac") : secret "dns-default-metrics-tls" not found Jul 06 22:47:05.542555 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:05.542520 2561 generic.go:356] "Generic (PLEG): container finished" podID="ad332d8f-fbd9-4a1d-8ff0-3473e9d50590" containerID="bc724b4807b0117588547a3053e9d50bf2ea9cc36168ea5f796260c34865b4be" exitCode=0 Jul 06 22:47:05.543227 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:05.542565 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerDied","Data":"bc724b4807b0117588547a3053e9d50bf2ea9cc36168ea5f796260c34865b4be"} Jul 06 22:47:06.546410 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:06.546374 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wtsvg" event={"ID":"efb1a3a3-9752-443b-834d-6ae1b3c0b448","Type":"ContainerStarted","Data":"33b010306bf3280d5b81f3e314df4b7213362a72b546c5b15af0cd7586300fa0"} Jul 06 22:47:06.549505 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:06.549479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" event={"ID":"ad332d8f-fbd9-4a1d-8ff0-3473e9d50590","Type":"ContainerStarted","Data":"be268d36636c50da56b70b22074052db1c0ce7f590dfb3a017152e42c10e4eb0"} Jul 06 22:47:06.561326 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:06.561278 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wtsvg" podStartSLOduration=1.152394327 podStartE2EDuration="3.561263761s" totalImagesPullingTime="2.408869434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:06.560036789 +0000 UTC m=+39.859738409" watchObservedRunningTime="2026-07-06 22:47:06.561263761 +0000 UTC m=+39.860965363" Jul 06 22:47:06.582652 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:06.582600 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bw8fc" podStartSLOduration=10.718359938 podStartE2EDuration="39.582583139s" totalImagesPullingTime="28.864223201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:06.581103899 +0000 UTC m=+39.880805500" watchObservedRunningTime="2026-07-06 22:47:06.582583139 +0000 UTC m=+39.882284741" Jul 06 22:47:06.793539 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:06.793503 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:06.793721 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:06.793635 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:47:06.793721 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:06.793715 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls podName:4d3599fe-dbea-4bdc-b913-846b419c5fac nodeName:}" failed. No retries permitted until 2026-07-06 22:47:10.793679138 +0000 UTC m=+44.093380718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls") pod "dns-default-8p8t2" (UID: "4d3599fe-dbea-4bdc-b913-846b419c5fac") : secret "dns-default-metrics-tls" not found Jul 06 22:47:07.986833 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:07.986800 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8"] Jul 06 22:47:08.019006 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.018964 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8"] Jul 06 22:47:08.019176 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.019076 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" Jul 06 22:47:08.021446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.021424 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:47:08.021533 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.021450 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7wkrv\"" Jul 06 22:47:08.022491 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.022473 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 22:47:08.094064 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.094028 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pj9wr"] Jul 06 22:47:08.101779 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.101742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd58p\" (UniqueName: \"kubernetes.io/projected/77391107-c4a1-4d4e-b602-21a6bfeb9c25-kube-api-access-fd58p\") pod \"volume-data-source-validator-5d456fdbcd-gq6q8\" (UID: \"77391107-c4a1-4d4e-b602-21a6bfeb9c25\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" Jul 06 22:47:08.108917 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.108890 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2"] Jul 06 22:47:08.109037 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.109006 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.112162 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.112138 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 22:47:08.112270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.112224 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 22:47:08.112348 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.112287 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 22:47:08.112348 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.112230 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ftx8p\"" Jul 06 22:47:08.113281 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.113260 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:47:08.117121 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.117103 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 22:47:08.122859 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.122840 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66bd778bf6-hkpss"] Jul 06 22:47:08.122963 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.122950 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:08.125083 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.125064 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 22:47:08.125369 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.125355 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:47:08.125463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.125387 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kk9pj\"" Jul 06 22:47:08.125463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.125411 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 22:47:08.144922 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.144893 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pj9wr"] Jul 06 22:47:08.144922 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.144923 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2"] Jul 06 22:47:08.145087 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.144954 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66bd778bf6-hkpss"] Jul 06 22:47:08.145087 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.145002 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.147426 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.147403 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 22:47:08.147526 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.147410 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 22:47:08.147902 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.147879 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 22:47:08.148623 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.148591 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qtpz\"" Jul 06 22:47:08.155427 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.155408 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 22:47:08.197083 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.197050 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5fcc5d685f-k7msg"] Jul 06 22:47:08.202309 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.202275 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd58p\" (UniqueName: \"kubernetes.io/projected/77391107-c4a1-4d4e-b602-21a6bfeb9c25-kube-api-access-fd58p\") pod \"volume-data-source-validator-5d456fdbcd-gq6q8\" (UID: \"77391107-c4a1-4d4e-b602-21a6bfeb9c25\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" Jul 06 22:47:08.202447 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.202317 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18094565-1360-46e6-9800-d2e14b59c959-trusted-ca\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.202447 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.202337 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzbk6\" (UniqueName: \"kubernetes.io/projected/18094565-1360-46e6-9800-d2e14b59c959-kube-api-access-zzbk6\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.202518 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.202473 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18094565-1360-46e6-9800-d2e14b59c959-config\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.202518 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.202502 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18094565-1360-46e6-9800-d2e14b59c959-serving-cert\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.212181 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.212152 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-9hzjz"] Jul 06 22:47:08.212341 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.212291 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.214844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.214820 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 22:47:08.214844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.214836 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 22:47:08.215236 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.215203 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 22:47:08.215341 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.215203 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 22:47:08.215341 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.215254 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 22:47:08.215341 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.215264 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zbk7j\"" Jul 06 22:47:08.215633 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.215618 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 22:47:08.220957 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.220933 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd58p\" (UniqueName: \"kubernetes.io/projected/77391107-c4a1-4d4e-b602-21a6bfeb9c25-kube-api-access-fd58p\") pod \"volume-data-source-validator-5d456fdbcd-gq6q8\" (UID: \"77391107-c4a1-4d4e-b602-21a6bfeb9c25\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" Jul 06 22:47:08.232276 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.232245 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-9hzjz"] Jul 06 22:47:08.232276 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.232282 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5fcc5d685f-k7msg"] Jul 06 22:47:08.232431 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.232332 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.234817 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.234795 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 22:47:08.234932 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.234794 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 22:47:08.235133 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.235111 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 22:47:08.235278 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.235261 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 22:47:08.235423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.235407 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5ggr4\"" Jul 06 22:47:08.240128 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.240108 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 22:47:08.303509 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303478 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-default-certificate\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.303509 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303513 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-bound-sa-token\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.303762 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303554 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18094565-1360-46e6-9800-d2e14b59c959-config\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.303762 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303578 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-trusted-ca\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.303762 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303595 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnsvp\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-kube-api-access-qnsvp\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.303762 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303614 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-stats-auth\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.303762 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18094565-1360-46e6-9800-d2e14b59c959-serving-cert\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.303935 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303770 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:08.303935 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303806 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxpb7\" (UniqueName: \"kubernetes.io/projected/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-kube-api-access-bxpb7\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:08.303935 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303834 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-image-registry-private-configuration\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.303935 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303862 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-ca-trust-extracted\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.303935 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303888 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-certificates\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.303935 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303912 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-installation-pull-secrets\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.304132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.303969 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.304132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.304011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.304132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.304050 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18094565-1360-46e6-9800-d2e14b59c959-trusted-ca\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.304132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.304074 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzbk6\" (UniqueName: \"kubernetes.io/projected/18094565-1360-46e6-9800-d2e14b59c959-kube-api-access-zzbk6\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.304132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.304112 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.304132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.304127 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8skc\" (UniqueName: \"kubernetes.io/projected/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-kube-api-access-p8skc\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.304841 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.304823 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18094565-1360-46e6-9800-d2e14b59c959-trusted-ca\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.305963 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.305946 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18094565-1360-46e6-9800-d2e14b59c959-serving-cert\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.312546 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.312525 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzbk6\" (UniqueName: \"kubernetes.io/projected/18094565-1360-46e6-9800-d2e14b59c959-kube-api-access-zzbk6\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.317849 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.317828 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18094565-1360-46e6-9800-d2e14b59c959-config\") pod \"console-operator-68b757fb67-pj9wr\" (UID: \"18094565-1360-46e6-9800-d2e14b59c959\") " pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.328465 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.328446 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" Jul 06 22:47:08.404579 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.404545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-installation-pull-secrets\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.404827 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.404801 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/22fb2b7b-2c50-42c5-96fd-b2325515019f-snapshots\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.404975 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.404960 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.405045 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405029 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnsvp\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-kube-api-access-qnsvp\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.405107 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405064 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.405159 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.405105 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:47:08.405216 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.405201 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:08.905177453 +0000 UTC m=+42.204879036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : secret "router-metrics-certs-default" not found Jul 06 22:47:08.405520 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405112 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-default-certificate\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.405605 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405544 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22fb2b7b-2c50-42c5-96fd-b2325515019f-serving-cert\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.405664 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405643 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpb7\" (UniqueName: \"kubernetes.io/projected/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-kube-api-access-bxpb7\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:08.405748 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22fb2b7b-2c50-42c5-96fd-b2325515019f-tmp\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.405748 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405721 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22fb2b7b-2c50-42c5-96fd-b2325515019f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.405843 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405751 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:08.405843 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405785 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-ca-trust-extracted\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.405843 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405818 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-certificates\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.405996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.405996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405871 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-bound-sa-token\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.405996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8skc\" (UniqueName: \"kubernetes.io/projected/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-kube-api-access-p8skc\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.405996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405955 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s222\" (UniqueName: \"kubernetes.io/projected/22fb2b7b-2c50-42c5-96fd-b2325515019f-kube-api-access-9s222\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.405996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.405980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-trusted-ca\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.406284 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.406009 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-stats-auth\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.406284 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.406056 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22fb2b7b-2c50-42c5-96fd-b2325515019f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.406284 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.406092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-image-registry-private-configuration\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.407551 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.406601 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:08.906578961 +0000 UTC m=+42.206280541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : configmap references non-existent config key: service-ca.crt Jul 06 22:47:08.407551 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.406977 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:47:08.407551 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.406997 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bd778bf6-hkpss: secret "image-registry-tls" not found Jul 06 22:47:08.407551 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.407165 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-certificates\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.407825 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.407639 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-ca-trust-extracted\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.408005 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.407979 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls podName:bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:08.907958402 +0000 UTC m=+42.207659993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls") pod "image-registry-66bd778bf6-hkpss" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb") : secret "image-registry-tls" not found Jul 06 22:47:08.408429 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.408296 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-default-certificate\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.408429 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.408335 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:47:08.408429 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.408408 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls podName:2e2495d9-94a5-4fc1-b91a-f42cf7e71112 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:08.908393 +0000 UTC m=+42.208094600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8skt2" (UID: "2e2495d9-94a5-4fc1-b91a-f42cf7e71112") : secret "samples-operator-tls" not found Jul 06 22:47:08.408641 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.408484 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-trusted-ca\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.408967 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.408944 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-installation-pull-secrets\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.410167 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.410140 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-image-registry-private-configuration\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.411192 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.411149 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-stats-auth\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.416635 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.416599 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-bound-sa-token\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.416909 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.416890 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8skc\" (UniqueName: \"kubernetes.io/projected/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-kube-api-access-p8skc\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.417122 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.417103 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnsvp\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-kube-api-access-qnsvp\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.417358 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.417340 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxpb7\" (UniqueName: \"kubernetes.io/projected/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-kube-api-access-bxpb7\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:08.419149 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.419133 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:08.469094 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.469009 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8"] Jul 06 22:47:08.472266 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:08.472233 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77391107_c4a1_4d4e_b602_21a6bfeb9c25.slice/crio-e9a0619b62b454c5bb17b6ce6c27a3df9779d53e5a8e4cab50de82fa80ca9419 WatchSource:0}: Error finding container e9a0619b62b454c5bb17b6ce6c27a3df9779d53e5a8e4cab50de82fa80ca9419: Status 404 returned error can't find the container with id e9a0619b62b454c5bb17b6ce6c27a3df9779d53e5a8e4cab50de82fa80ca9419 Jul 06 22:47:08.507255 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.507170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22fb2b7b-2c50-42c5-96fd-b2325515019f-tmp\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.507255 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.507206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22fb2b7b-2c50-42c5-96fd-b2325515019f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.507440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.507300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s222\" (UniqueName: \"kubernetes.io/projected/22fb2b7b-2c50-42c5-96fd-b2325515019f-kube-api-access-9s222\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.507440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.507348 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22fb2b7b-2c50-42c5-96fd-b2325515019f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.507440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.507372 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/22fb2b7b-2c50-42c5-96fd-b2325515019f-snapshots\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.507569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.507442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22fb2b7b-2c50-42c5-96fd-b2325515019f-serving-cert\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.508360 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.507958 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/22fb2b7b-2c50-42c5-96fd-b2325515019f-tmp\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.508360 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.508209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22fb2b7b-2c50-42c5-96fd-b2325515019f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.508529 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.508489 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/22fb2b7b-2c50-42c5-96fd-b2325515019f-snapshots\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.508899 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.508874 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22fb2b7b-2c50-42c5-96fd-b2325515019f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.511568 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.511127 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22fb2b7b-2c50-42c5-96fd-b2325515019f-serving-cert\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.528917 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.528888 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s222\" (UniqueName: \"kubernetes.io/projected/22fb2b7b-2c50-42c5-96fd-b2325515019f-kube-api-access-9s222\") pod \"insights-operator-5c847dcfc4-9hzjz\" (UID: \"22fb2b7b-2c50-42c5-96fd-b2325515019f\") " pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.541731 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.541698 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" Jul 06 22:47:08.556235 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.556198 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" event={"ID":"77391107-c4a1-4d4e-b602-21a6bfeb9c25","Type":"ContainerStarted","Data":"e9a0619b62b454c5bb17b6ce6c27a3df9779d53e5a8e4cab50de82fa80ca9419"} Jul 06 22:47:08.556401 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.556385 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pj9wr"] Jul 06 22:47:08.559383 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:08.559362 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18094565_1360_46e6_9800_d2e14b59c959.slice/crio-009ad1fc96952d789a1d37f55180c2d378c5b9036a590d73f57b79ffbf24980f WatchSource:0}: Error finding container 009ad1fc96952d789a1d37f55180c2d378c5b9036a590d73f57b79ffbf24980f: Status 404 returned error can't find the container with id 009ad1fc96952d789a1d37f55180c2d378c5b9036a590d73f57b79ffbf24980f Jul 06 22:47:08.700400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.700367 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-9hzjz"] Jul 06 22:47:08.703522 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:08.703497 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22fb2b7b_2c50_42c5_96fd_b2325515019f.slice/crio-19fd600971ecad3e1d9809843ac3a6c1b58ffda3df708c0731008ee5f50088b0 WatchSource:0}: Error finding container 19fd600971ecad3e1d9809843ac3a6c1b58ffda3df708c0731008ee5f50088b0: Status 404 returned error can't find the container with id 19fd600971ecad3e1d9809843ac3a6c1b58ffda3df708c0731008ee5f50088b0 Jul 06 22:47:08.910043 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.909942 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:08.910043 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.910018 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.910043 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.910039 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:08.910097 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910099 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910120 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bd778bf6-hkpss: secret "image-registry-tls" not found Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910122 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910175 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910189 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls podName:bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:09.91016879 +0000 UTC m=+43.209870388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls") pod "image-registry-66bd778bf6-hkpss" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb") : secret "image-registry-tls" not found Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910209 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls podName:2e2495d9-94a5-4fc1-b91a-f42cf7e71112 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:09.910198334 +0000 UTC m=+43.209899914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8skt2" (UID: "2e2495d9-94a5-4fc1-b91a-f42cf7e71112") : secret "samples-operator-tls" not found Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910221 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:09.910215676 +0000 UTC m=+43.209917256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : secret "router-metrics-certs-default" not found Jul 06 22:47:08.910292 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:08.910231 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:09.910226177 +0000 UTC m=+43.209927757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : configmap references non-existent config key: service-ca.crt Jul 06 22:47:09.561064 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:09.560979 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" event={"ID":"18094565-1360-46e6-9800-d2e14b59c959","Type":"ContainerStarted","Data":"009ad1fc96952d789a1d37f55180c2d378c5b9036a590d73f57b79ffbf24980f"} Jul 06 22:47:09.562773 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:09.562739 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" event={"ID":"22fb2b7b-2c50-42c5-96fd-b2325515019f","Type":"ContainerStarted","Data":"19fd600971ecad3e1d9809843ac3a6c1b58ffda3df708c0731008ee5f50088b0"} Jul 06 22:47:09.920832 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:09.920803 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:09.921009 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:09.920843 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:09.921009 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:09.920890 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:09.921009 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:09.920908 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:09.921009 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.920958 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:47:09.921195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.921017 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:11.921002064 +0000 UTC m=+45.220703644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : configmap references non-existent config key: service-ca.crt Jul 06 22:47:09.921195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.921029 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:47:09.921195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.921038 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:47:09.921195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.921059 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bd778bf6-hkpss: secret "image-registry-tls" not found Jul 06 22:47:09.921195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.921032 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls podName:2e2495d9-94a5-4fc1-b91a-f42cf7e71112 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:11.921025277 +0000 UTC m=+45.220726857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8skt2" (UID: "2e2495d9-94a5-4fc1-b91a-f42cf7e71112") : secret "samples-operator-tls" not found Jul 06 22:47:09.921195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.921083 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:11.9210698 +0000 UTC m=+45.220771380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : secret "router-metrics-certs-default" not found Jul 06 22:47:09.921195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:09.921111 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls podName:bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:11.921094486 +0000 UTC m=+45.220796084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls") pod "image-registry-66bd778bf6-hkpss" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb") : secret "image-registry-tls" not found Jul 06 22:47:10.827908 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:10.827872 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:10.828324 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:10.828025 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:47:10.828324 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:10.828086 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls podName:4d3599fe-dbea-4bdc-b913-846b419c5fac nodeName:}" failed. No retries permitted until 2026-07-06 22:47:18.828069735 +0000 UTC m=+52.127771324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls") pod "dns-default-8p8t2" (UID: "4d3599fe-dbea-4bdc-b913-846b419c5fac") : secret "dns-default-metrics-tls" not found Jul 06 22:47:11.937021 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:11.936978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:11.937021 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:11.937025 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:11.937139 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:11.937185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937238 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.937212126 +0000 UTC m=+49.236913719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : configmap references non-existent config key: service-ca.crt Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937238 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937267 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937276 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937280 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bd778bf6-hkpss: secret "image-registry-tls" not found Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937326 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls podName:2e2495d9-94a5-4fc1-b91a-f42cf7e71112 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.937312838 +0000 UTC m=+49.237014422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8skt2" (UID: "2e2495d9-94a5-4fc1-b91a-f42cf7e71112") : secret "samples-operator-tls" not found Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937360 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls podName:bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.937349731 +0000 UTC m=+49.237051324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls") pod "image-registry-66bd778bf6-hkpss" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb") : secret "image-registry-tls" not found Jul 06 22:47:11.937542 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:11.937373 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.937366515 +0000 UTC m=+49.237068095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : secret "router-metrics-certs-default" not found Jul 06 22:47:12.572173 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.572128 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" event={"ID":"22fb2b7b-2c50-42c5-96fd-b2325515019f","Type":"ContainerStarted","Data":"217fd5302b4bb97db32d9191d9e64113c464b4454ec6b37ef3f49544b6cc370a"} Jul 06 22:47:12.573814 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.573782 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" event={"ID":"77391107-c4a1-4d4e-b602-21a6bfeb9c25","Type":"ContainerStarted","Data":"42a986e1b96342753dcaa35d6289d7b9364a821002e0351a002130dc740ef49a"} Jul 06 22:47:12.575493 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.575470 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/0.log" Jul 06 22:47:12.575608 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.575505 2561 generic.go:356] "Generic (PLEG): container finished" podID="18094565-1360-46e6-9800-d2e14b59c959" containerID="1f1c294ce84f70560bb4ac3b21b93892c68f04d05e385e8fb2006062eb6772c7" exitCode=255 Jul 06 22:47:12.575608 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.575528 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" event={"ID":"18094565-1360-46e6-9800-d2e14b59c959","Type":"ContainerDied","Data":"1f1c294ce84f70560bb4ac3b21b93892c68f04d05e385e8fb2006062eb6772c7"} Jul 06 22:47:12.575793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.575778 2561 scope.go:117] "RemoveContainer" containerID="1f1c294ce84f70560bb4ac3b21b93892c68f04d05e385e8fb2006062eb6772c7" Jul 06 22:47:12.604111 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.604019 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" podStartSLOduration=1.6274688880000001 podStartE2EDuration="4.603997001s" totalImagesPullingTime="2.976528113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:12.588573024 +0000 UTC m=+45.888274627" watchObservedRunningTime="2026-07-06 22:47:12.603997001 +0000 UTC m=+45.903698608" Jul 06 22:47:12.604861 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:12.604823 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gq6q8" podStartSLOduration=2.433476699 podStartE2EDuration="5.604807255s" totalImagesPullingTime="3.171330556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:12.603965984 +0000 UTC m=+45.903667610" watchObservedRunningTime="2026-07-06 22:47:12.604807255 +0000 UTC m=+45.904508858" Jul 06 22:47:13.536165 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.536127 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dz8m4"] Jul 06 22:47:13.557939 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.557910 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dz8m4"] Jul 06 22:47:13.558091 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.558035 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.560291 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.560267 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 22:47:13.560426 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.560355 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 22:47:13.560508 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.560494 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xdg5g\"" Jul 06 22:47:13.579542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.579514 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/1.log" Jul 06 22:47:13.579969 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.579951 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/0.log" Jul 06 22:47:13.580071 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.579983 2561 generic.go:356] "Generic (PLEG): container finished" podID="18094565-1360-46e6-9800-d2e14b59c959" containerID="323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90" exitCode=255 Jul 06 22:47:13.580112 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.580069 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" event={"ID":"18094565-1360-46e6-9800-d2e14b59c959","Type":"ContainerDied","Data":"323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90"} Jul 06 22:47:13.580155 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.580124 2561 scope.go:117] "RemoveContainer" containerID="1f1c294ce84f70560bb4ac3b21b93892c68f04d05e385e8fb2006062eb6772c7" Jul 06 22:47:13.580447 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.580431 2561 scope.go:117] "RemoveContainer" containerID="323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90" Jul 06 22:47:13.580662 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:13.580628 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pj9wr_openshift-console-operator(18094565-1360-46e6-9800-d2e14b59c959)\"" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" podUID="18094565-1360-46e6-9800-d2e14b59c959" Jul 06 22:47:13.753492 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.753452 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-data-volume\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.753767 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.753738 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-crio-socket\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.753836 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.753783 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dhnn\" (UniqueName: \"kubernetes.io/projected/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-api-access-4dhnn\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.753972 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.753945 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.754033 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.753998 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.855303 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.855226 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.855303 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.855267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.855303 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.855296 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-data-volume\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.855506 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:13.855375 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:47:13.855506 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:13.855396 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:47:13.855506 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:13.855453 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:14.355433105 +0000 UTC m=+47.655134689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : secret "insights-runtime-extractor-tls" not found Jul 06 22:47:13.855506 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:13.855478 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:14.355466126 +0000 UTC m=+47.655167705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : configmap "kube-rbac-proxy" not found Jul 06 22:47:13.855655 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.855551 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-crio-socket\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.855655 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.855559 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-data-volume\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.855655 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.855579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dhnn\" (UniqueName: \"kubernetes.io/projected/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-api-access-4dhnn\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.855786 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.855729 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-crio-socket\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:13.864287 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:13.864256 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dhnn\" (UniqueName: \"kubernetes.io/projected/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-api-access-4dhnn\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:14.359236 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:14.359199 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:14.359236 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:14.359242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:14.359465 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:14.359344 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:47:14.359465 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:14.359426 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:47:14.359465 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:14.359428 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.359405725 +0000 UTC m=+48.659107310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : configmap "kube-rbac-proxy" not found Jul 06 22:47:14.359565 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:14.359476 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:15.359462938 +0000 UTC m=+48.659164522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : secret "insights-runtime-extractor-tls" not found Jul 06 22:47:14.584445 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:14.584416 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/1.log" Jul 06 22:47:14.584943 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:14.584867 2561 scope.go:117] "RemoveContainer" containerID="323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90" Jul 06 22:47:14.585067 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:14.585045 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pj9wr_openshift-console-operator(18094565-1360-46e6-9800-d2e14b59c959)\"" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" podUID="18094565-1360-46e6-9800-d2e14b59c959" Jul 06 22:47:15.129040 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:15.129009 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lgv7m_4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3/dns-node-resolver/0.log" Jul 06 22:47:15.369958 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:15.369914 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:15.370146 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:15.369956 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:15.370146 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.370039 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:47:15.370146 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.370092 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:47:15.370146 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.370107 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:17.370090381 +0000 UTC m=+50.669791962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : configmap "kube-rbac-proxy" not found Jul 06 22:47:15.370146 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.370131 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:17.370120007 +0000 UTC m=+50.669821587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : secret "insights-runtime-extractor-tls" not found Jul 06 22:47:15.975233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:15.975190 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:15.975233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:15.975234 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:15.975299 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:15.975319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975345 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975422 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:23.975404689 +0000 UTC m=+57.275106269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : secret "router-metrics-certs-default" not found Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975437 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:23.975431036 +0000 UTC m=+57.275132616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : configmap references non-existent config key: service-ca.crt Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975454 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975469 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bd778bf6-hkpss: secret "image-registry-tls" not found Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975506 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls podName:bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:23.975491196 +0000 UTC m=+57.275192803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls") pod "image-registry-66bd778bf6-hkpss" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb") : secret "image-registry-tls" not found Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975457 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:47:15.975708 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:15.975549 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls podName:2e2495d9-94a5-4fc1-b91a-f42cf7e71112 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:23.975539565 +0000 UTC m=+57.275241145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-8skt2" (UID: "2e2495d9-94a5-4fc1-b91a-f42cf7e71112") : secret "samples-operator-tls" not found Jul 06 22:47:16.335508 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.335436 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qlk8d_3eb86e34-7ac4-49e6-910d-a04169181619/node-ca/0.log" Jul 06 22:47:16.489990 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.489958 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-dc455"] Jul 06 22:47:16.545007 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.544971 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-dc455"] Jul 06 22:47:16.545169 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.545082 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.547588 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.547557 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 22:47:16.547745 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.547651 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6762d\"" Jul 06 22:47:16.547745 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.547652 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 22:47:16.548442 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.548424 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 22:47:16.548533 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.548460 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 22:47:16.579723 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.579670 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/43bb6adf-96f2-4892-a5a4-0a003d70a662-signing-key\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.579899 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.579756 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/43bb6adf-96f2-4892-a5a4-0a003d70a662-signing-cabundle\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.579899 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.579790 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvh6b\" (UniqueName: \"kubernetes.io/projected/43bb6adf-96f2-4892-a5a4-0a003d70a662-kube-api-access-jvh6b\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.680898 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.680868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/43bb6adf-96f2-4892-a5a4-0a003d70a662-signing-key\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.681059 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.680913 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/43bb6adf-96f2-4892-a5a4-0a003d70a662-signing-cabundle\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.681059 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.680935 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvh6b\" (UniqueName: \"kubernetes.io/projected/43bb6adf-96f2-4892-a5a4-0a003d70a662-kube-api-access-jvh6b\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.681552 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.681525 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/43bb6adf-96f2-4892-a5a4-0a003d70a662-signing-cabundle\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.683341 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.683318 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/43bb6adf-96f2-4892-a5a4-0a003d70a662-signing-key\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.688989 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.688959 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvh6b\" (UniqueName: \"kubernetes.io/projected/43bb6adf-96f2-4892-a5a4-0a003d70a662-kube-api-access-jvh6b\") pod \"service-ca-758f599c5-dc455\" (UID: \"43bb6adf-96f2-4892-a5a4-0a003d70a662\") " pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.853992 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.853950 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-dc455" Jul 06 22:47:16.929433 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:16.929398 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wtsvg_efb1a3a3-9752-443b-834d-6ae1b3c0b448/serve-healthcheck-canary/0.log" Jul 06 22:47:17.000669 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:17.000635 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-dc455"] Jul 06 22:47:17.003678 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:17.003651 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bb6adf_96f2_4892_a5a4_0a003d70a662.slice/crio-4d0c23da06a6e1e53ad702b4f514899c6dd4b3f67f701ead127e96d406afea07 WatchSource:0}: Error finding container 4d0c23da06a6e1e53ad702b4f514899c6dd4b3f67f701ead127e96d406afea07: Status 404 returned error can't find the container with id 4d0c23da06a6e1e53ad702b4f514899c6dd4b3f67f701ead127e96d406afea07 Jul 06 22:47:17.386790 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:17.386701 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:17.386790 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:17.386745 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:17.386970 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:17.386847 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:47:17.386970 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:17.386932 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:21.386909562 +0000 UTC m=+54.686611167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : configmap "kube-rbac-proxy" not found Jul 06 22:47:17.386970 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:17.386934 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:47:17.387080 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:17.386978 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:21.386966364 +0000 UTC m=+54.686667955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : secret "insights-runtime-extractor-tls" not found Jul 06 22:47:17.594553 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:17.594514 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-dc455" event={"ID":"43bb6adf-96f2-4892-a5a4-0a003d70a662","Type":"ContainerStarted","Data":"4d0c23da06a6e1e53ad702b4f514899c6dd4b3f67f701ead127e96d406afea07"} Jul 06 22:47:18.419283 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:18.419247 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:18.419781 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:18.419391 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:18.419781 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:18.419768 2561 scope.go:117] "RemoveContainer" containerID="323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90" Jul 06 22:47:18.419974 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:18.419952 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pj9wr_openshift-console-operator(18094565-1360-46e6-9800-d2e14b59c959)\"" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" podUID="18094565-1360-46e6-9800-d2e14b59c959" Jul 06 22:47:18.597239 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:18.597208 2561 scope.go:117] "RemoveContainer" containerID="323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90" Jul 06 22:47:18.597410 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:18.597383 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pj9wr_openshift-console-operator(18094565-1360-46e6-9800-d2e14b59c959)\"" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" podUID="18094565-1360-46e6-9800-d2e14b59c959" Jul 06 22:47:18.903394 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:18.903357 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:18.903651 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:18.903524 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:47:18.903651 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:18.903605 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls podName:4d3599fe-dbea-4bdc-b913-846b419c5fac nodeName:}" failed. No retries permitted until 2026-07-06 22:47:34.903583987 +0000 UTC m=+68.203285582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls") pod "dns-default-8p8t2" (UID: "4d3599fe-dbea-4bdc-b913-846b419c5fac") : secret "dns-default-metrics-tls" not found Jul 06 22:47:19.604860 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:19.604824 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-dc455" event={"ID":"43bb6adf-96f2-4892-a5a4-0a003d70a662","Type":"ContainerStarted","Data":"be3ec3faa08240f9e5d3d229e52a444cd1b0e2d45856b807ed2d96083a2a0b3c"} Jul 06 22:47:19.621653 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:19.621594 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-dc455" podStartSLOduration=1.462468178 podStartE2EDuration="3.621576883s" totalImagesPullingTime="2.159108705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:19.620432902 +0000 UTC m=+52.920134505" watchObservedRunningTime="2026-07-06 22:47:19.621576883 +0000 UTC m=+52.921278501" Jul 06 22:47:21.429275 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:21.429220 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:21.429275 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:21.429280 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:21.429929 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:21.429361 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:47:21.429929 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:21.429451 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:29.429428136 +0000 UTC m=+62.729129719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : configmap "kube-rbac-proxy" not found Jul 06 22:47:21.429929 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:21.429468 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:47:21.429929 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:21.429535 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:29.429518367 +0000 UTC m=+62.729219953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : secret "insights-runtime-extractor-tls" not found Jul 06 22:47:24.055838 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.055783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.055850 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:24.055933 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.055938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:24.056015 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:40.055996836 +0000 UTC m=+73.355698419 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : secret "router-metrics-certs-default" not found Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:24.056062 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle podName:d70e9c41-fd49-4be6-91a4-a0f0f1fafba2 nodeName:}" failed. No retries permitted until 2026-07-06 22:47:40.056039697 +0000 UTC m=+73.355741299 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle") pod "router-default-5fcc5d685f-k7msg" (UID: "d70e9c41-fd49-4be6-91a4-a0f0f1fafba2") : configmap references non-existent config key: service-ca.crt Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.056095 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") pod \"image-registry-66bd778bf6-hkpss\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:24.056223 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:24.056233 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66bd778bf6-hkpss: secret "image-registry-tls" not found Jul 06 22:47:24.056318 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:24.056265 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls podName:bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:40.056256859 +0000 UTC m=+73.355958443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls") pod "image-registry-66bd778bf6-hkpss" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb") : secret "image-registry-tls" not found Jul 06 22:47:24.058288 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.058265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2495d9-94a5-4fc1-b91a-f42cf7e71112-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-8skt2\" (UID: \"2e2495d9-94a5-4fc1-b91a-f42cf7e71112\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:24.331753 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.331659 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" Jul 06 22:47:24.454435 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.454399 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2"] Jul 06 22:47:24.623496 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:24.623409 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" event={"ID":"2e2495d9-94a5-4fc1-b91a-f42cf7e71112","Type":"ContainerStarted","Data":"c3ea29e52e6e118f37a0a81cf4fe8916cf7f9b3bda9d2b1901d96d3e18728fc4"} Jul 06 22:47:26.632333 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:26.632287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" event={"ID":"2e2495d9-94a5-4fc1-b91a-f42cf7e71112","Type":"ContainerStarted","Data":"6324da644955127bbc85a5890580ce540864517b1d01d2e850fed88adecd92d9"} Jul 06 22:47:26.632748 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:26.632350 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" event={"ID":"2e2495d9-94a5-4fc1-b91a-f42cf7e71112","Type":"ContainerStarted","Data":"66ded0b44feb036760128626418717c9650ef00c505be9f36590619e7006709b"} Jul 06 22:47:26.649796 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:26.649734 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-8skt2" podStartSLOduration=17.262253259 podStartE2EDuration="18.649719369s" totalImagesPullingTime="1.38746611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:26.647952515 +0000 UTC m=+59.947654117" watchObservedRunningTime="2026-07-06 22:47:26.649719369 +0000 UTC m=+59.949421028" Jul 06 22:47:29.506703 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:29.506646 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:29.506703 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:29.506706 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:29.507307 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:29.506811 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:47:29.507307 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:29.506863 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls podName:a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb nodeName:}" failed. No retries permitted until 2026-07-06 22:47:45.506850746 +0000 UTC m=+78.806552326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dz8m4" (UID: "a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb") : secret "insights-runtime-extractor-tls" not found Jul 06 22:47:29.507406 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:29.507308 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:29.524669 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:29.524639 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sprk8" Jul 06 22:47:32.214584 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:32.214554 2561 scope.go:117] "RemoveContainer" containerID="323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90" Jul 06 22:47:32.658187 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:32.658113 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 22:47:32.658530 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:32.658512 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/1.log" Jul 06 22:47:32.658579 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:32.658548 2561 generic.go:356] "Generic (PLEG): container finished" podID="18094565-1360-46e6-9800-d2e14b59c959" containerID="20702e6ece45ce8f64dc104cf4c223d222ce4be3d8afa725ba3ca356588dd0ff" exitCode=255 Jul 06 22:47:32.658613 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:32.658587 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" event={"ID":"18094565-1360-46e6-9800-d2e14b59c959","Type":"ContainerDied","Data":"20702e6ece45ce8f64dc104cf4c223d222ce4be3d8afa725ba3ca356588dd0ff"} Jul 06 22:47:32.658654 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:32.658631 2561 scope.go:117] "RemoveContainer" containerID="323ecb14cf6a79a19811e4278ea66ef73bfc18f595f9ca3dab8543033bf54d90" Jul 06 22:47:32.658980 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:32.658959 2561 scope.go:117] "RemoveContainer" containerID="20702e6ece45ce8f64dc104cf4c223d222ce4be3d8afa725ba3ca356588dd0ff" Jul 06 22:47:32.659167 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:32.659147 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-pj9wr_openshift-console-operator(18094565-1360-46e6-9800-d2e14b59c959)\"" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" podUID="18094565-1360-46e6-9800-d2e14b59c959" Jul 06 22:47:33.136648 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.136608 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:47:33.139242 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.139217 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:47:33.149386 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.149353 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5422cceb-c1fe-4581-847a-59883489e159-metrics-certs\") pod \"network-metrics-daemon-82jtz\" (UID: \"5422cceb-c1fe-4581-847a-59883489e159\") " pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:47:33.237565 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.237527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:33.237998 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.237590 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:47:33.240202 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.240176 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:47:33.240202 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.240195 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 22:47:33.250554 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.250531 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44-original-pull-secret\") pod \"global-pull-secret-syncer-qsln2\" (UID: \"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44\") " pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:47:33.250699 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.250674 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 22:47:33.261097 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.261067 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cprnr\" (UniqueName: \"kubernetes.io/projected/6e2aa88e-8b84-4997-acc8-162e6ce24a45-kube-api-access-cprnr\") pod \"network-check-target-c884g\" (UID: \"6e2aa88e-8b84-4997-acc8-162e6ce24a45\") " pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:33.329669 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.329628 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmht5\"" Jul 06 22:47:33.333742 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.333719 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qsln2" Jul 06 22:47:33.337248 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.337224 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-82jtz" Jul 06 22:47:33.343517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.343496 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j9ddx\"" Jul 06 22:47:33.351817 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.351772 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:33.491722 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.491675 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-82jtz"] Jul 06 22:47:33.494347 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:33.494317 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5422cceb_c1fe_4581_847a_59883489e159.slice/crio-4444edbeec0cd3b945e6192338bbadd8949ffbd37569db1cee79dd3751c68c44 WatchSource:0}: Error finding container 4444edbeec0cd3b945e6192338bbadd8949ffbd37569db1cee79dd3751c68c44: Status 404 returned error can't find the container with id 4444edbeec0cd3b945e6192338bbadd8949ffbd37569db1cee79dd3751c68c44 Jul 06 22:47:33.664262 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.664183 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 22:47:33.665352 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.665328 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-82jtz" event={"ID":"5422cceb-c1fe-4581-847a-59883489e159","Type":"ContainerStarted","Data":"4444edbeec0cd3b945e6192338bbadd8949ffbd37569db1cee79dd3751c68c44"} Jul 06 22:47:33.716200 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.716166 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c884g"] Jul 06 22:47:33.718345 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:33.718319 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qsln2"] Jul 06 22:47:33.719061 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:33.719031 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2aa88e_8b84_4997_acc8_162e6ce24a45.slice/crio-5573a642215fc45eb737cbaa8341a844c41604323fb927737815cab4f213bdec WatchSource:0}: Error finding container 5573a642215fc45eb737cbaa8341a844c41604323fb927737815cab4f213bdec: Status 404 returned error can't find the container with id 5573a642215fc45eb737cbaa8341a844c41604323fb927737815cab4f213bdec Jul 06 22:47:33.721393 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:33.721267 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc293b72_c4f9_46bc_a0ae_ceb0f0cefe44.slice/crio-cd1ca80e3a8808ed98fe5d0f78347f07ddbb1efdb80966843005dfa7a791d484 WatchSource:0}: Error finding container cd1ca80e3a8808ed98fe5d0f78347f07ddbb1efdb80966843005dfa7a791d484: Status 404 returned error can't find the container with id cd1ca80e3a8808ed98fe5d0f78347f07ddbb1efdb80966843005dfa7a791d484 Jul 06 22:47:34.672457 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:34.672371 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c884g" event={"ID":"6e2aa88e-8b84-4997-acc8-162e6ce24a45","Type":"ContainerStarted","Data":"5573a642215fc45eb737cbaa8341a844c41604323fb927737815cab4f213bdec"} Jul 06 22:47:34.673983 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:34.673949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qsln2" event={"ID":"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44","Type":"ContainerStarted","Data":"cd1ca80e3a8808ed98fe5d0f78347f07ddbb1efdb80966843005dfa7a791d484"} Jul 06 22:47:34.954463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:34.954363 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:34.957432 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:34.957408 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d3599fe-dbea-4bdc-b913-846b419c5fac-metrics-tls\") pod \"dns-default-8p8t2\" (UID: \"4d3599fe-dbea-4bdc-b913-846b419c5fac\") " pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:35.176887 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:35.176854 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ctk6j\"" Jul 06 22:47:35.184350 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:35.184314 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:35.372868 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:35.372822 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8p8t2"] Jul 06 22:47:35.376163 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:35.376128 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3599fe_dbea_4bdc_b913_846b419c5fac.slice/crio-acd9dee96327ca776cd273c1cf0535034b93bc697a6aa7fd268ba96c44b49545 WatchSource:0}: Error finding container acd9dee96327ca776cd273c1cf0535034b93bc697a6aa7fd268ba96c44b49545: Status 404 returned error can't find the container with id acd9dee96327ca776cd273c1cf0535034b93bc697a6aa7fd268ba96c44b49545 Jul 06 22:47:35.685609 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:35.682523 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-82jtz" event={"ID":"5422cceb-c1fe-4581-847a-59883489e159","Type":"ContainerStarted","Data":"5f9948a9c2fea21c31ff18a22bb56a0bdbc05e8e09571dcf06f74dfaaa4e4693"} Jul 06 22:47:35.685609 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:35.682638 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-82jtz" event={"ID":"5422cceb-c1fe-4581-847a-59883489e159","Type":"ContainerStarted","Data":"d05cd5dc56b03e3bb0931ff592e391c2555f7e704c1046db4bf0e67987794e58"} Jul 06 22:47:35.687785 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:35.687742 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8p8t2" event={"ID":"4d3599fe-dbea-4bdc-b913-846b419c5fac","Type":"ContainerStarted","Data":"acd9dee96327ca776cd273c1cf0535034b93bc697a6aa7fd268ba96c44b49545"} Jul 06 22:47:35.700322 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:35.700231 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-82jtz" podStartSLOduration=67.005145089 podStartE2EDuration="1m8.700207303s" totalImagesPullingTime="1.695062214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:35.698860094 +0000 UTC m=+68.998561699" watchObservedRunningTime="2026-07-06 22:47:35.700207303 +0000 UTC m=+68.999908907" Jul 06 22:47:36.487884 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.487794 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4"] Jul 06 22:47:36.506081 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.506045 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66bd778bf6-hkpss"] Jul 06 22:47:36.506267 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.506108 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4"] Jul 06 22:47:36.506267 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.506233 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.506373 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:36.506318 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" podUID="bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" Jul 06 22:47:36.509997 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.509950 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 22:47:36.510256 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.510237 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pg49w\"" Jul 06 22:47:36.510672 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.510456 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 22:47:36.510804 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.510737 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 22:47:36.511636 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.510943 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 22:47:36.527667 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.527635 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65c97fdb9c-875fz"] Jul 06 22:47:36.549509 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.549482 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.554293 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.554229 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c97fdb9c-875fz"] Jul 06 22:47:36.596102 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.595666 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8"] Jul 06 22:47:36.618556 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.618523 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8"] Jul 06 22:47:36.618752 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.618646 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:36.622210 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.622084 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 22:47:36.622210 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.622185 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sbtqs\"" Jul 06 22:47:36.622210 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.622197 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 22:47:36.669786 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.669747 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f86563b7-0cc6-4b18-842d-66751fc0ecb9-registry-certificates\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.669971 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.669841 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4cd\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-kube-api-access-8z4cd\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.669971 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.669864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f86563b7-0cc6-4b18-842d-66751fc0ecb9-trusted-ca\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.669971 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.669889 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-bound-sa-token\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.670131 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.670016 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f86563b7-0cc6-4b18-842d-66751fc0ecb9-image-registry-private-configuration\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.670131 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.670050 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f86563b7-0cc6-4b18-842d-66751fc0ecb9-installation-pull-secrets\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.670131 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.670087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv74f\" (UniqueName: \"kubernetes.io/projected/96599f9e-f191-4692-b2f0-cbf348d4dad3-kube-api-access-rv74f\") pod \"managed-serviceaccount-addon-agent-76bdb4764-54qq4\" (UID: \"96599f9e-f191-4692-b2f0-cbf348d4dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.670131 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.670113 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f86563b7-0cc6-4b18-842d-66751fc0ecb9-ca-trust-extracted\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.670320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.670141 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96599f9e-f191-4692-b2f0-cbf348d4dad3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76bdb4764-54qq4\" (UID: \"96599f9e-f191-4692-b2f0-cbf348d4dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.670320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.670241 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-registry-tls\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.692156 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.692122 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:36.706276 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.706247 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:36.771069 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.770982 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-trusted-ca\") pod \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " Jul 06 22:47:36.771069 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771062 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-ca-trust-extracted\") pod \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " Jul 06 22:47:36.771296 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771100 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-installation-pull-secrets\") pod \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " Jul 06 22:47:36.771296 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771224 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-image-registry-private-configuration\") pod \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " Jul 06 22:47:36.771408 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771329 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-bound-sa-token\") pod \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " Jul 06 22:47:36.771408 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771371 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnsvp\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-kube-api-access-qnsvp\") pod \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " Jul 06 22:47:36.771408 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771380 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:47:36.771408 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771406 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-certificates\") pod \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\" (UID: \"bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb\") " Jul 06 22:47:36.771606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771509 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:47:36.771606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4cd\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-kube-api-access-8z4cd\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.771724 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771622 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f86563b7-0cc6-4b18-842d-66751fc0ecb9-trusted-ca\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.771724 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-bound-sa-token\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.771832 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771741 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f86563b7-0cc6-4b18-842d-66751fc0ecb9-image-registry-private-configuration\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.771832 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771769 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f86563b7-0cc6-4b18-842d-66751fc0ecb9-installation-pull-secrets\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.771832 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/77db57d6-78b8-43c9-9ce2-7e8e045f4377-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-jpkh8\" (UID: \"77db57d6-78b8-43c9-9ce2-7e8e045f4377\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:36.771832 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv74f\" (UniqueName: \"kubernetes.io/projected/96599f9e-f191-4692-b2f0-cbf348d4dad3-kube-api-access-rv74f\") pod \"managed-serviceaccount-addon-agent-76bdb4764-54qq4\" (UID: \"96599f9e-f191-4692-b2f0-cbf348d4dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.772044 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771846 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f86563b7-0cc6-4b18-842d-66751fc0ecb9-ca-trust-extracted\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.772044 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771872 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96599f9e-f191-4692-b2f0-cbf348d4dad3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76bdb4764-54qq4\" (UID: \"96599f9e-f191-4692-b2f0-cbf348d4dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.772044 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771943 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-registry-tls\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.772044 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771937 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:47:36.772044 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.771987 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f86563b7-0cc6-4b18-842d-66751fc0ecb9-registry-certificates\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.772281 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.772042 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77db57d6-78b8-43c9-9ce2-7e8e045f4377-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jpkh8\" (UID: \"77db57d6-78b8-43c9-9ce2-7e8e045f4377\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:36.772281 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.772137 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-ca-trust-extracted\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:36.772281 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.772156 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-certificates\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:36.772281 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.772174 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-trusted-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:36.773063 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.772326 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f86563b7-0cc6-4b18-842d-66751fc0ecb9-ca-trust-extracted\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.773149 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.773122 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f86563b7-0cc6-4b18-842d-66751fc0ecb9-registry-certificates\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.773605 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.773586 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f86563b7-0cc6-4b18-842d-66751fc0ecb9-trusted-ca\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.774441 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.774406 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-kube-api-access-qnsvp" (OuterVolumeSpecName: "kube-api-access-qnsvp") pod "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb"). InnerVolumeSpecName "kube-api-access-qnsvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:47:36.775220 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.775185 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:47:36.775537 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.775512 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:47:36.775795 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.775753 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" (UID: "bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:47:36.776294 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.776263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f86563b7-0cc6-4b18-842d-66751fc0ecb9-installation-pull-secrets\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.776466 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.776437 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96599f9e-f191-4692-b2f0-cbf348d4dad3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76bdb4764-54qq4\" (UID: \"96599f9e-f191-4692-b2f0-cbf348d4dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.777152 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.777118 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-registry-tls\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.777672 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.777650 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f86563b7-0cc6-4b18-842d-66751fc0ecb9-image-registry-private-configuration\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.792469 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.792439 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-bound-sa-token\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.792596 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.792575 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4cd\" (UniqueName: \"kubernetes.io/projected/f86563b7-0cc6-4b18-842d-66751fc0ecb9-kube-api-access-8z4cd\") pod \"image-registry-65c97fdb9c-875fz\" (UID: \"f86563b7-0cc6-4b18-842d-66751fc0ecb9\") " pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.794764 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.794738 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv74f\" (UniqueName: \"kubernetes.io/projected/96599f9e-f191-4692-b2f0-cbf348d4dad3-kube-api-access-rv74f\") pod \"managed-serviceaccount-addon-agent-76bdb4764-54qq4\" (UID: \"96599f9e-f191-4692-b2f0-cbf348d4dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.827672 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.827633 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" Jul 06 22:47:36.865268 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.865239 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qtpz\"" Jul 06 22:47:36.872657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.872616 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77db57d6-78b8-43c9-9ce2-7e8e045f4377-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jpkh8\" (UID: \"77db57d6-78b8-43c9-9ce2-7e8e045f4377\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:36.872823 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.872757 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/77db57d6-78b8-43c9-9ce2-7e8e045f4377-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-jpkh8\" (UID: \"77db57d6-78b8-43c9-9ce2-7e8e045f4377\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:36.872823 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.872819 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-installation-pull-secrets\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:36.872933 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.872836 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-image-registry-private-configuration\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:36.872933 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.872853 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-bound-sa-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:36.872933 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.872867 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnsvp\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-kube-api-access-qnsvp\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:36.873137 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.873115 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:36.873575 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.873548 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/77db57d6-78b8-43c9-9ce2-7e8e045f4377-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-jpkh8\" (UID: \"77db57d6-78b8-43c9-9ce2-7e8e045f4377\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:36.875342 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.875315 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77db57d6-78b8-43c9-9ce2-7e8e045f4377-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-jpkh8\" (UID: \"77db57d6-78b8-43c9-9ce2-7e8e045f4377\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:36.932409 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:36.932379 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" Jul 06 22:47:37.695575 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:37.695542 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66bd778bf6-hkpss" Jul 06 22:47:37.726439 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:37.726404 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66bd778bf6-hkpss"] Jul 06 22:47:37.730811 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:37.730771 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66bd778bf6-hkpss"] Jul 06 22:47:37.883028 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:37.882989 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb-registry-tls\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:47:38.419548 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:38.419509 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:38.419743 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:38.419598 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:38.420021 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:38.420000 2561 scope.go:117] "RemoveContainer" containerID="20702e6ece45ce8f64dc104cf4c223d222ce4be3d8afa725ba3ca356588dd0ff" Jul 06 22:47:38.420195 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:38.420174 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-pj9wr_openshift-console-operator(18094565-1360-46e6-9800-d2e14b59c959)\"" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" podUID="18094565-1360-46e6-9800-d2e14b59c959" Jul 06 22:47:39.217759 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.213968 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4"] Jul 06 22:47:39.234413 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:39.234372 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96599f9e_f191_4692_b2f0_cbf348d4dad3.slice/crio-9660bafa394f4c42cff1722ee7ff1a44bcc34696bc3effbab2792f65b1a1b35f WatchSource:0}: Error finding container 9660bafa394f4c42cff1722ee7ff1a44bcc34696bc3effbab2792f65b1a1b35f: Status 404 returned error can't find the container with id 9660bafa394f4c42cff1722ee7ff1a44bcc34696bc3effbab2792f65b1a1b35f Jul 06 22:47:39.238443 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.238398 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb" path="/var/lib/kubelet/pods/bbf44be9-c7fb-4825-bdf5-4d44c0a2ebeb/volumes" Jul 06 22:47:39.248679 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.248650 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c97fdb9c-875fz"] Jul 06 22:47:39.254826 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:39.254794 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf86563b7_0cc6_4b18_842d_66751fc0ecb9.slice/crio-ffe621eb49c5c12ef3423589e7aeb8ba430f5a86c6a9306f8131c647c1b10e4e WatchSource:0}: Error finding container ffe621eb49c5c12ef3423589e7aeb8ba430f5a86c6a9306f8131c647c1b10e4e: Status 404 returned error can't find the container with id ffe621eb49c5c12ef3423589e7aeb8ba430f5a86c6a9306f8131c647c1b10e4e Jul 06 22:47:39.266920 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.266838 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8"] Jul 06 22:47:39.272051 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:39.272006 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77db57d6_78b8_43c9_9ce2_7e8e045f4377.slice/crio-7ba97bef835376c824fbbc3f13e73a84097c88c396225e1b1dc1402aa9315575 WatchSource:0}: Error finding container 7ba97bef835376c824fbbc3f13e73a84097c88c396225e1b1dc1402aa9315575: Status 404 returned error can't find the container with id 7ba97bef835376c824fbbc3f13e73a84097c88c396225e1b1dc1402aa9315575 Jul 06 22:47:39.706887 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.706598 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c884g" event={"ID":"6e2aa88e-8b84-4997-acc8-162e6ce24a45","Type":"ContainerStarted","Data":"91eba3df8d9fd62741b9e3fe1ce3810883a9a077f8a868a36386513ebb663853"} Jul 06 22:47:39.706887 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.706718 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:47:39.708275 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.708246 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qsln2" event={"ID":"fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44","Type":"ContainerStarted","Data":"3a33aea9c4e3f891536c2d84304def7edaf39948b49747c278c157ca76097f57"} Jul 06 22:47:39.710587 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.710464 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8p8t2" event={"ID":"4d3599fe-dbea-4bdc-b913-846b419c5fac","Type":"ContainerStarted","Data":"23138850277d9ba7be64e5bbf70ed11c911ec364e7b2f28a3aa726525cb74709"} Jul 06 22:47:39.710587 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.710527 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8p8t2" event={"ID":"4d3599fe-dbea-4bdc-b913-846b419c5fac","Type":"ContainerStarted","Data":"ab02dc88f31669f1b4adb96c899b5764dc6fe77485af251377b9b22870fabb80"} Jul 06 22:47:39.710587 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.710556 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:39.711848 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.711797 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" event={"ID":"77db57d6-78b8-43c9-9ce2-7e8e045f4377","Type":"ContainerStarted","Data":"7ba97bef835376c824fbbc3f13e73a84097c88c396225e1b1dc1402aa9315575"} Jul 06 22:47:39.713094 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.713069 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" event={"ID":"96599f9e-f191-4692-b2f0-cbf348d4dad3","Type":"ContainerStarted","Data":"9660bafa394f4c42cff1722ee7ff1a44bcc34696bc3effbab2792f65b1a1b35f"} Jul 06 22:47:39.715556 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.715520 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" event={"ID":"f86563b7-0cc6-4b18-842d-66751fc0ecb9","Type":"ContainerStarted","Data":"1612c0bcf1de0287ba66b01925b6aaf8dac1025ceafbc8c1f9fd176a0dad4041"} Jul 06 22:47:39.715705 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.715564 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" event={"ID":"f86563b7-0cc6-4b18-842d-66751fc0ecb9","Type":"ContainerStarted","Data":"ffe621eb49c5c12ef3423589e7aeb8ba430f5a86c6a9306f8131c647c1b10e4e"} Jul 06 22:47:39.715773 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.715709 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:47:39.723640 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.723581 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c884g" podStartSLOduration=67.402989115 podStartE2EDuration="1m12.723560965s" totalImagesPullingTime="5.32057185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:39.722730785 +0000 UTC m=+73.022432391" watchObservedRunningTime="2026-07-06 22:47:39.723560965 +0000 UTC m=+73.023262568" Jul 06 22:47:39.738122 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.738060 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qsln2" podStartSLOduration=67.412040633 podStartE2EDuration="1m12.738041657s" totalImagesPullingTime="5.326001024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:39.737227442 +0000 UTC m=+73.036929048" watchObservedRunningTime="2026-07-06 22:47:39.738041657 +0000 UTC m=+73.037743262" Jul 06 22:47:39.754377 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.754308 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8p8t2" podStartSLOduration=33.096646684 podStartE2EDuration="36.754287727s" totalImagesPullingTime="3.657641043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:39.754258391 +0000 UTC m=+73.053960013" watchObservedRunningTime="2026-07-06 22:47:39.754287727 +0000 UTC m=+73.053989334" Jul 06 22:47:39.777483 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:39.777422 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" podStartSLOduration=3.777398325 podStartE2EDuration="3.777398325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:39.77730191 +0000 UTC m=+73.077003539" watchObservedRunningTime="2026-07-06 22:47:39.777398325 +0000 UTC m=+73.077099929" Jul 06 22:47:40.105918 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.105830 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:40.105918 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.105870 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:40.106549 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.106518 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-service-ca-bundle\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:40.108384 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.108355 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d70e9c41-fd49-4be6-91a4-a0f0f1fafba2-metrics-certs\") pod \"router-default-5fcc5d685f-k7msg\" (UID: \"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2\") " pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:40.329019 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.328925 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zbk7j\"" Jul 06 22:47:40.336979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.336932 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:40.497294 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.497135 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5fcc5d685f-k7msg"] Jul 06 22:47:40.500430 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:40.500397 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70e9c41_fd49_4be6_91a4_a0f0f1fafba2.slice/crio-44012e8c91c50d03fa3906d3950eccb77fe26bcc9181911724894d43ba730457 WatchSource:0}: Error finding container 44012e8c91c50d03fa3906d3950eccb77fe26bcc9181911724894d43ba730457: Status 404 returned error can't find the container with id 44012e8c91c50d03fa3906d3950eccb77fe26bcc9181911724894d43ba730457 Jul 06 22:47:40.722752 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.722633 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" event={"ID":"77db57d6-78b8-43c9-9ce2-7e8e045f4377","Type":"ContainerStarted","Data":"1ed6f375da29aa680c3598a692d04cbc8de10fdee60292b536910d3aef9473d4"} Jul 06 22:47:40.725788 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.725755 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" event={"ID":"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2","Type":"ContainerStarted","Data":"1293dd39f21116706b17413b28369914c1513ea43efb26152f9fe86ad9639bdc"} Jul 06 22:47:40.725938 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.725804 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" event={"ID":"d70e9c41-fd49-4be6-91a4-a0f0f1fafba2","Type":"ContainerStarted","Data":"44012e8c91c50d03fa3906d3950eccb77fe26bcc9181911724894d43ba730457"} Jul 06 22:47:40.758127 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.758064 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" podStartSLOduration=32.758046772 podStartE2EDuration="32.758046772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:40.757028582 +0000 UTC m=+74.056730200" watchObservedRunningTime="2026-07-06 22:47:40.758046772 +0000 UTC m=+74.057748374" Jul 06 22:47:40.759292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:40.758985 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-jpkh8" podStartSLOduration=3.802858232 podStartE2EDuration="4.758965873s" totalImagesPullingTime="956.107641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:40.738967191 +0000 UTC m=+74.038668794" watchObservedRunningTime="2026-07-06 22:47:40.758965873 +0000 UTC m=+74.058667474" Jul 06 22:47:41.337568 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:41.337525 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:41.340398 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:41.340372 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:41.730340 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:41.730305 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:41.731668 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:41.731639 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5fcc5d685f-k7msg" Jul 06 22:47:42.736222 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:42.736182 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" event={"ID":"96599f9e-f191-4692-b2f0-cbf348d4dad3","Type":"ContainerStarted","Data":"40622618e44f928123d7899ccfd165ee1ec8054e27f07fe63aa0af0928e5e786"} Jul 06 22:47:45.551289 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:45.551233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:45.553598 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:45.553572 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dz8m4\" (UID: \"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb\") " pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:45.670009 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:45.669970 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xdg5g\"" Jul 06 22:47:45.677941 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:45.677909 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dz8m4" Jul 06 22:47:45.812626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:45.812515 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76bdb4764-54qq4" podStartSLOduration=6.990996759 podStartE2EDuration="9.812490814s" totalImagesPullingTime="2.821494055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:42.752287048 +0000 UTC m=+76.051988651" watchObservedRunningTime="2026-07-06 22:47:45.812490814 +0000 UTC m=+79.112192417" Jul 06 22:47:45.813191 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:45.813171 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dz8m4"] Jul 06 22:47:45.815637 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:45.815608 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dcb7e8_2f91_4e33_a0cc_12e9f929b7eb.slice/crio-084b6a6e552d1b69d4ea6e508faaaea4499a10905cc133dab90a20dcec532268 WatchSource:0}: Error finding container 084b6a6e552d1b69d4ea6e508faaaea4499a10905cc133dab90a20dcec532268: Status 404 returned error can't find the container with id 084b6a6e552d1b69d4ea6e508faaaea4499a10905cc133dab90a20dcec532268 Jul 06 22:47:46.753732 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:46.753674 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dz8m4" event={"ID":"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb","Type":"ContainerStarted","Data":"084b6a6e552d1b69d4ea6e508faaaea4499a10905cc133dab90a20dcec532268"} Jul 06 22:47:49.728334 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:49.728300 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8p8t2" Jul 06 22:47:49.770007 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:49.769912 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dz8m4" event={"ID":"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb","Type":"ContainerStarted","Data":"50049931bb74d92757d65378811f9ad4bcd2920fa20ad6f4a26eb24babb71db0"} Jul 06 22:47:50.775764 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:50.775723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dz8m4" event={"ID":"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb","Type":"ContainerStarted","Data":"18f294bc7ab3ff227ffba5eb49de740fc785731dd486279e9718743eb4eae3e6"} Jul 06 22:47:52.250725 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.249804 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-khkw7"] Jul 06 22:47:52.265010 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.262252 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.267764 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.267506 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 22:47:52.270065 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.269445 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 22:47:52.270065 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.269754 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 22:47:52.270065 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.269943 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 22:47:52.270065 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.269954 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 22:47:52.271649 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.270579 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-njcl6\"" Jul 06 22:47:52.271649 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.270823 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.405836 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-wtmp\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.405894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-root\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.405921 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jv7\" (UniqueName: \"kubernetes.io/projected/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-kube-api-access-k4jv7\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.406005 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-textfile\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.406030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-accelerators-collector-config\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.406061 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-metrics-client-ca\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.406096 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.406121 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-sys\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.406678 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.406167 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.507998 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-root\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508050 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jv7\" (UniqueName: \"kubernetes.io/projected/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-kube-api-access-k4jv7\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508125 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-textfile\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508149 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-accelerators-collector-config\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508183 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-metrics-client-ca\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508222 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508246 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-sys\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508270 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.508317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-wtmp\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.509070 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-accelerators-collector-config\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.509245 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-root\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.509820 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-textfile\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.510328 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-sys\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:52.510411 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:52.510464 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls podName:b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c nodeName:}" failed. No retries permitted until 2026-07-06 22:47:53.010446756 +0000 UTC m=+86.310148341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls") pod "node-exporter-khkw7" (UID: "b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c") : secret "node-exporter-tls" not found Jul 06 22:47:52.511144 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.511051 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-metrics-client-ca\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.512132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.512092 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-wtmp\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.515409 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.515362 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:52.522903 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:52.522843 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jv7\" (UniqueName: \"kubernetes.io/projected/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-kube-api-access-k4jv7\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:53.013460 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.013415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:53.013643 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:53.013573 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 22:47:53.013643 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:47:53.013639 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls podName:b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c nodeName:}" failed. No retries permitted until 2026-07-06 22:47:54.013623753 +0000 UTC m=+87.313325332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls") pod "node-exporter-khkw7" (UID: "b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c") : secret "node-exporter-tls" not found Jul 06 22:47:53.214702 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.214656 2561 scope.go:117] "RemoveContainer" containerID="20702e6ece45ce8f64dc104cf4c223d222ce4be3d8afa725ba3ca356588dd0ff" Jul 06 22:47:53.795258 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.795216 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dz8m4" event={"ID":"a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb","Type":"ContainerStarted","Data":"c881e64cdbc146f61372a6d743783ea9ad7d18641910f739373d9c5afa568fad"} Jul 06 22:47:53.797186 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.797163 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 22:47:53.797306 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.797253 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" event={"ID":"18094565-1360-46e6-9800-d2e14b59c959","Type":"ContainerStarted","Data":"445dcd6df6f0b4fc5dff418c1e4f0a2979b432326e0f9c7d99a59ba3ec57d207"} Jul 06 22:47:53.797511 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.797494 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:53.819613 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.819549 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dz8m4" podStartSLOduration=33.836571891 podStartE2EDuration="40.819529508s" totalImagesPullingTime="6.982957617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:47:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:53.816794535 +0000 UTC m=+87.116496163" watchObservedRunningTime="2026-07-06 22:47:53.819529508 +0000 UTC m=+87.119231111" Jul 06 22:47:53.834044 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:53.833987 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" podStartSLOduration=42.718010295 podStartE2EDuration="45.833967345s" totalImagesPullingTime="3.11595705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:53.83375889 +0000 UTC m=+87.133460523" watchObservedRunningTime="2026-07-06 22:47:53.833967345 +0000 UTC m=+87.133668947" Jul 06 22:47:54.022842 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:54.022799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:54.025217 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:54.025187 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c-node-exporter-tls\") pod \"node-exporter-khkw7\" (UID: \"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c\") " pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:54.078391 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:54.078286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-khkw7" Jul 06 22:47:54.089366 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:47:54.089332 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e39119_24e6_44c0_8cbc_f2f4c4c06d0c.slice/crio-fe95ec3595fb3519fd7a9fbdce71717445a7ddbadfd669375c753e3f94a9050d WatchSource:0}: Error finding container fe95ec3595fb3519fd7a9fbdce71717445a7ddbadfd669375c753e3f94a9050d: Status 404 returned error can't find the container with id fe95ec3595fb3519fd7a9fbdce71717445a7ddbadfd669375c753e3f94a9050d Jul 06 22:47:54.642921 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:54.642888 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-pj9wr" Jul 06 22:47:54.803370 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:54.803306 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-khkw7" event={"ID":"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c","Type":"ContainerStarted","Data":"fe95ec3595fb3519fd7a9fbdce71717445a7ddbadfd669375c753e3f94a9050d"} Jul 06 22:47:55.808402 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:55.808367 2561 generic.go:356] "Generic (PLEG): container finished" podID="b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c" containerID="2a12e12908bf69d79b631579ff979a9aa134f9eaf95b20a0c07dfbc63d6f5720" exitCode=0 Jul 06 22:47:55.808878 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:55.808450 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-khkw7" event={"ID":"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c","Type":"ContainerDied","Data":"2a12e12908bf69d79b631579ff979a9aa134f9eaf95b20a0c07dfbc63d6f5720"} Jul 06 22:47:56.814305 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:56.814266 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-khkw7" event={"ID":"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c","Type":"ContainerStarted","Data":"0b22c4304cda9abd62c46aae70439fe0089cd43dc4d5d506439dc800741d0036"} Jul 06 22:47:56.814714 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:56.814332 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-khkw7" event={"ID":"b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c","Type":"ContainerStarted","Data":"d0f03c4fa1267df3e9fc18ccfdcd6fbaec77c86279d414bb3318ee485f1327da"} Jul 06 22:47:56.834010 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:56.833938 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-khkw7" podStartSLOduration=3.903384396 podStartE2EDuration="4.833920578s" totalImagesPullingTime="864.397516ms" totalInitContainerRuntime="66.138666ms" isStatefulPod=true podCreationTimestamp="2026-07-06 22:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:56.832021504 +0000 UTC m=+90.131723109" watchObservedRunningTime="2026-07-06 22:47:56.833920578 +0000 UTC m=+90.133622241" Jul 06 22:47:59.936460 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.936419 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fc9dcfc7c-22v9j"] Jul 06 22:47:59.939565 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.939542 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:47:59.942106 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.942077 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 22:47:59.942232 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.942134 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 22:47:59.942232 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.942152 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 22:47:59.943302 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.943278 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-clp94\"" Jul 06 22:47:59.943423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.943354 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 22:47:59.943423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.943364 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 22:47:59.943423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.943355 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 22:47:59.943423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.943403 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 22:47:59.949101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.949077 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc9dcfc7c-22v9j"] Jul 06 22:47:59.963639 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.963607 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-serving-cert\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:47:59.963793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.963641 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-oauth-config\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:47:59.963793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.963668 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-service-ca\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:47:59.963793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.963704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-config\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:47:59.963793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.963769 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rxxl\" (UniqueName: \"kubernetes.io/projected/39ce7c5c-b698-4a12-a7d2-384f82160b11-kube-api-access-8rxxl\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:47:59.963925 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:47:59.963827 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-oauth-serving-cert\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065066 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065029 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-serving-cert\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065258 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065071 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-oauth-config\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065258 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065100 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-service-ca\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065258 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065118 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-config\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rxxl\" (UniqueName: \"kubernetes.io/projected/39ce7c5c-b698-4a12-a7d2-384f82160b11-kube-api-access-8rxxl\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065314 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-oauth-serving-cert\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065915 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065886 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-service-ca\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.065915 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065904 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-config\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.066047 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.065948 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-oauth-serving-cert\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.067564 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.067543 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-oauth-config\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.067641 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.067612 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-serving-cert\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.073959 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.073934 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rxxl\" (UniqueName: \"kubernetes.io/projected/39ce7c5c-b698-4a12-a7d2-384f82160b11-kube-api-access-8rxxl\") pod \"console-6fc9dcfc7c-22v9j\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.248853 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.248759 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:00.381745 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.381721 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fc9dcfc7c-22v9j"] Jul 06 22:48:00.384298 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:48:00.384271 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ce7c5c_b698_4a12_a7d2_384f82160b11.slice/crio-587675413fca363cd2f3d1ce1e6a10e2007c29da6dee2516f2c111a1f7ff8815 WatchSource:0}: Error finding container 587675413fca363cd2f3d1ce1e6a10e2007c29da6dee2516f2c111a1f7ff8815: Status 404 returned error can't find the container with id 587675413fca363cd2f3d1ce1e6a10e2007c29da6dee2516f2c111a1f7ff8815 Jul 06 22:48:00.730716 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.730669 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65c97fdb9c-875fz" Jul 06 22:48:00.832645 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:00.832604 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc9dcfc7c-22v9j" event={"ID":"39ce7c5c-b698-4a12-a7d2-384f82160b11","Type":"ContainerStarted","Data":"587675413fca363cd2f3d1ce1e6a10e2007c29da6dee2516f2c111a1f7ff8815"} Jul 06 22:48:03.848234 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:03.848198 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc9dcfc7c-22v9j" event={"ID":"39ce7c5c-b698-4a12-a7d2-384f82160b11","Type":"ContainerStarted","Data":"638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f"} Jul 06 22:48:03.865836 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:03.865778 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fc9dcfc7c-22v9j" podStartSLOduration=2.250688475 podStartE2EDuration="4.865761529s" totalImagesPullingTime="2.615073054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:48:03.864672665 +0000 UTC m=+97.164374286" watchObservedRunningTime="2026-07-06 22:48:03.865761529 +0000 UTC m=+97.165463131" Jul 06 22:48:07.724613 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.724570 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f56f85544-hbjqd"] Jul 06 22:48:07.729428 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.729404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.737312 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.737279 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 22:48:07.738410 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.738385 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f56f85544-hbjqd"] Jul 06 22:48:07.830101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.830056 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-serving-cert\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.830101 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.830095 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-oauth-config\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.830320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.830115 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-service-ca\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.830320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.830150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-oauth-serving-cert\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.830320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.830242 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-console-config\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.830320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.830274 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-trusted-ca-bundle\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.830320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.830304 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjdbq\" (UniqueName: \"kubernetes.io/projected/877427df-588c-42f1-939a-988655d54a86-kube-api-access-vjdbq\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.930742 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.930673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-serving-cert\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.930927 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.930750 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-oauth-config\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.930927 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.930776 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-service-ca\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.930927 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.930809 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-oauth-serving-cert\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.930927 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.930887 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-console-config\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.930927 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.930913 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-trusted-ca-bundle\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.931179 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.930950 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjdbq\" (UniqueName: \"kubernetes.io/projected/877427df-588c-42f1-939a-988655d54a86-kube-api-access-vjdbq\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.931676 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.931646 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-service-ca\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.931806 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.931717 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-oauth-serving-cert\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.931806 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.931726 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-console-config\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.931989 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.931968 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-trusted-ca-bundle\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.933293 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.933268 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-oauth-config\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.933425 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.933406 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-serving-cert\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:07.942146 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:07.942119 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjdbq\" (UniqueName: \"kubernetes.io/projected/877427df-588c-42f1-939a-988655d54a86-kube-api-access-vjdbq\") pod \"console-5f56f85544-hbjqd\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:08.038986 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:08.038878 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:08.175743 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:08.175715 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f56f85544-hbjqd"] Jul 06 22:48:08.178111 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:48:08.178085 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877427df_588c_42f1_939a_988655d54a86.slice/crio-f9d2e122c7c867fe06bc0fd162dad82e9e0d8a08b503db955c1b4a823530d1bb WatchSource:0}: Error finding container f9d2e122c7c867fe06bc0fd162dad82e9e0d8a08b503db955c1b4a823530d1bb: Status 404 returned error can't find the container with id f9d2e122c7c867fe06bc0fd162dad82e9e0d8a08b503db955c1b4a823530d1bb Jul 06 22:48:08.868134 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:08.868092 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f56f85544-hbjqd" event={"ID":"877427df-588c-42f1-939a-988655d54a86","Type":"ContainerStarted","Data":"9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506"} Jul 06 22:48:08.868546 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:08.868156 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f56f85544-hbjqd" event={"ID":"877427df-588c-42f1-939a-988655d54a86","Type":"ContainerStarted","Data":"f9d2e122c7c867fe06bc0fd162dad82e9e0d8a08b503db955c1b4a823530d1bb"} Jul 06 22:48:08.884758 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:08.884678 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f56f85544-hbjqd" podStartSLOduration=1.884658959 podStartE2EDuration="1.884658959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:48:08.884283389 +0000 UTC m=+102.183985009" watchObservedRunningTime="2026-07-06 22:48:08.884658959 +0000 UTC m=+102.184360561" Jul 06 22:48:10.249614 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:10.249557 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:10.250039 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:10.249663 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:10.255305 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:10.255279 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:10.728388 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:10.728356 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c884g" Jul 06 22:48:10.879952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:10.879925 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:17.902548 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:17.902503 2561 generic.go:356] "Generic (PLEG): container finished" podID="22fb2b7b-2c50-42c5-96fd-b2325515019f" containerID="217fd5302b4bb97db32d9191d9e64113c464b4454ec6b37ef3f49544b6cc370a" exitCode=0 Jul 06 22:48:17.902955 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:17.902575 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" event={"ID":"22fb2b7b-2c50-42c5-96fd-b2325515019f","Type":"ContainerDied","Data":"217fd5302b4bb97db32d9191d9e64113c464b4454ec6b37ef3f49544b6cc370a"} Jul 06 22:48:17.902955 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:17.902947 2561 scope.go:117] "RemoveContainer" containerID="217fd5302b4bb97db32d9191d9e64113c464b4454ec6b37ef3f49544b6cc370a" Jul 06 22:48:18.039594 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:18.039492 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:18.039594 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:18.039561 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:18.044907 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:18.044877 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:18.907852 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:18.907805 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9hzjz" event={"ID":"22fb2b7b-2c50-42c5-96fd-b2325515019f","Type":"ContainerStarted","Data":"61eb9edc0d4203e13155674588e08de05538dc4e883844d496e2ad6e92619460"} Jul 06 22:48:18.911760 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:18.911736 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:48:18.974105 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:18.974073 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc9dcfc7c-22v9j"] Jul 06 22:48:44.000889 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.000830 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6fc9dcfc7c-22v9j" podUID="39ce7c5c-b698-4a12-a7d2-384f82160b11" containerName="console" containerID="cri-o://638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f" gracePeriod=15 Jul 06 22:48:44.267377 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.267349 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:44.442541 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.442501 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rxxl\" (UniqueName: \"kubernetes.io/projected/39ce7c5c-b698-4a12-a7d2-384f82160b11-kube-api-access-8rxxl\") pod \"39ce7c5c-b698-4a12-a7d2-384f82160b11\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " Jul 06 22:48:44.442541 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.442549 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-service-ca\") pod \"39ce7c5c-b698-4a12-a7d2-384f82160b11\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " Jul 06 22:48:44.442788 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.442600 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-oauth-serving-cert\") pod \"39ce7c5c-b698-4a12-a7d2-384f82160b11\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " Jul 06 22:48:44.442788 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.442763 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-oauth-config\") pod \"39ce7c5c-b698-4a12-a7d2-384f82160b11\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " Jul 06 22:48:44.442886 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.442816 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-serving-cert\") pod \"39ce7c5c-b698-4a12-a7d2-384f82160b11\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " Jul 06 22:48:44.442886 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.442864 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-config\") pod \"39ce7c5c-b698-4a12-a7d2-384f82160b11\" (UID: \"39ce7c5c-b698-4a12-a7d2-384f82160b11\") " Jul 06 22:48:44.443034 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.443011 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-service-ca" (OuterVolumeSpecName: "service-ca") pod "39ce7c5c-b698-4a12-a7d2-384f82160b11" (UID: "39ce7c5c-b698-4a12-a7d2-384f82160b11"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:48:44.443034 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.443025 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "39ce7c5c-b698-4a12-a7d2-384f82160b11" (UID: "39ce7c5c-b698-4a12-a7d2-384f82160b11"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:48:44.443157 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.443140 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:48:44.443217 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.443164 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:48:44.443333 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.443311 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-config" (OuterVolumeSpecName: "console-config") pod "39ce7c5c-b698-4a12-a7d2-384f82160b11" (UID: "39ce7c5c-b698-4a12-a7d2-384f82160b11"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:48:44.445037 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.445007 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "39ce7c5c-b698-4a12-a7d2-384f82160b11" (UID: "39ce7c5c-b698-4a12-a7d2-384f82160b11"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:48:44.445150 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.445014 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ce7c5c-b698-4a12-a7d2-384f82160b11-kube-api-access-8rxxl" (OuterVolumeSpecName: "kube-api-access-8rxxl") pod "39ce7c5c-b698-4a12-a7d2-384f82160b11" (UID: "39ce7c5c-b698-4a12-a7d2-384f82160b11"). InnerVolumeSpecName "kube-api-access-8rxxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:48:44.445150 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.445069 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "39ce7c5c-b698-4a12-a7d2-384f82160b11" (UID: "39ce7c5c-b698-4a12-a7d2-384f82160b11"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:48:44.543813 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.543717 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:48:44.543813 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.543758 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:48:44.543813 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.543768 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39ce7c5c-b698-4a12-a7d2-384f82160b11-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:48:44.543813 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:44.543776 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rxxl\" (UniqueName: \"kubernetes.io/projected/39ce7c5c-b698-4a12-a7d2-384f82160b11-kube-api-access-8rxxl\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:48:45.008418 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.008379 2561 generic.go:356] "Generic (PLEG): container finished" podID="39ce7c5c-b698-4a12-a7d2-384f82160b11" containerID="638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f" exitCode=0 Jul 06 22:48:45.008903 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.008427 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fc9dcfc7c-22v9j" Jul 06 22:48:45.008903 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.008446 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc9dcfc7c-22v9j" event={"ID":"39ce7c5c-b698-4a12-a7d2-384f82160b11","Type":"ContainerDied","Data":"638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f"} Jul 06 22:48:45.008903 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.008499 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fc9dcfc7c-22v9j" event={"ID":"39ce7c5c-b698-4a12-a7d2-384f82160b11","Type":"ContainerDied","Data":"587675413fca363cd2f3d1ce1e6a10e2007c29da6dee2516f2c111a1f7ff8815"} Jul 06 22:48:45.008903 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.008530 2561 scope.go:117] "RemoveContainer" containerID="638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f" Jul 06 22:48:45.017705 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.017668 2561 scope.go:117] "RemoveContainer" containerID="638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f" Jul 06 22:48:45.018014 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:48:45.017994 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f\": container with ID starting with 638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f not found: ID does not exist" containerID="638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f" Jul 06 22:48:45.018083 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.018021 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f"} err="failed to get container status \"638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f\": rpc error: code = NotFound desc = could not find container \"638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f\": container with ID starting with 638f014afb5f531d55b4d9f3c0133e306ad2748101953232ddec6db8b36b881f not found: ID does not exist" Jul 06 22:48:45.031646 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.031611 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fc9dcfc7c-22v9j"] Jul 06 22:48:45.034984 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.034956 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fc9dcfc7c-22v9j"] Jul 06 22:48:45.219402 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:48:45.219364 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ce7c5c-b698-4a12-a7d2-384f82160b11" path="/var/lib/kubelet/pods/39ce7c5c-b698-4a12-a7d2-384f82160b11/volumes" Jul 06 22:49:19.953513 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:19.953426 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-696c85d7db-kc8vn"] Jul 06 22:49:19.954135 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:19.953755 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ce7c5c-b698-4a12-a7d2-384f82160b11" containerName="console" Jul 06 22:49:19.954135 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:19.953770 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ce7c5c-b698-4a12-a7d2-384f82160b11" containerName="console" Jul 06 22:49:19.954135 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:19.953821 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ce7c5c-b698-4a12-a7d2-384f82160b11" containerName="console" Jul 06 22:49:19.956843 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:19.956820 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:19.968471 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:19.968439 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696c85d7db-kc8vn"] Jul 06 22:49:20.028065 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.028017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6b66\" (UniqueName: \"kubernetes.io/projected/b46b9d10-5f7c-4436-84ab-4e15db16ce37-kube-api-access-j6b66\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.028270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.028102 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-serving-cert\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.028270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.028128 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-config\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.028270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.028156 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-oauth-config\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.028270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.028178 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-trusted-ca-bundle\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.028270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.028205 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-service-ca\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.028270 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.028257 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-oauth-serving-cert\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.129209 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.129167 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-oauth-serving-cert\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.129414 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.129236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6b66\" (UniqueName: \"kubernetes.io/projected/b46b9d10-5f7c-4436-84ab-4e15db16ce37-kube-api-access-j6b66\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.129414 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.129285 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-serving-cert\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.129414 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.129309 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-config\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.129414 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.129334 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-oauth-config\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.129414 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.129357 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-trusted-ca-bundle\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.129414 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.129383 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-service-ca\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.130103 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.130070 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-oauth-serving-cert\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.130234 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.130070 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-config\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.130234 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.130152 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-service-ca\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.130234 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.130212 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-trusted-ca-bundle\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.131780 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.131749 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-oauth-config\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.132022 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.132003 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-serving-cert\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.136941 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.136918 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6b66\" (UniqueName: \"kubernetes.io/projected/b46b9d10-5f7c-4436-84ab-4e15db16ce37-kube-api-access-j6b66\") pod \"console-696c85d7db-kc8vn\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.268072 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.267972 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:20.407780 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:20.407742 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696c85d7db-kc8vn"] Jul 06 22:49:20.410844 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:49:20.410814 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb46b9d10_5f7c_4436_84ab_4e15db16ce37.slice/crio-b5897fe61d03c2cb2af65a4c4a94ce59449aafa47f11924ce2489e52b70609d1 WatchSource:0}: Error finding container b5897fe61d03c2cb2af65a4c4a94ce59449aafa47f11924ce2489e52b70609d1: Status 404 returned error can't find the container with id b5897fe61d03c2cb2af65a4c4a94ce59449aafa47f11924ce2489e52b70609d1 Jul 06 22:49:21.145271 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:21.145234 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c85d7db-kc8vn" event={"ID":"b46b9d10-5f7c-4436-84ab-4e15db16ce37","Type":"ContainerStarted","Data":"a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931"} Jul 06 22:49:21.145677 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:21.145285 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c85d7db-kc8vn" event={"ID":"b46b9d10-5f7c-4436-84ab-4e15db16ce37","Type":"ContainerStarted","Data":"b5897fe61d03c2cb2af65a4c4a94ce59449aafa47f11924ce2489e52b70609d1"} Jul 06 22:49:21.162351 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:21.162291 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-696c85d7db-kc8vn" podStartSLOduration=2.162274117 podStartE2EDuration="2.162274117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:49:21.16103956 +0000 UTC m=+174.460741194" watchObservedRunningTime="2026-07-06 22:49:21.162274117 +0000 UTC m=+174.461975786" Jul 06 22:49:30.269174 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:30.269125 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:30.269593 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:30.269196 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:30.274254 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:30.274222 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:31.185275 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:31.185243 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:49:31.229545 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:31.229507 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f56f85544-hbjqd"] Jul 06 22:49:56.258752 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.258654 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5f56f85544-hbjqd" podUID="877427df-588c-42f1-939a-988655d54a86" containerName="console" containerID="cri-o://9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506" gracePeriod=15 Jul 06 22:49:56.502022 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.501992 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:49:56.634292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634182 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-console-config\") pod \"877427df-588c-42f1-939a-988655d54a86\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " Jul 06 22:49:56.634292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634264 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-trusted-ca-bundle\") pod \"877427df-588c-42f1-939a-988655d54a86\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " Jul 06 22:49:56.634548 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634404 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjdbq\" (UniqueName: \"kubernetes.io/projected/877427df-588c-42f1-939a-988655d54a86-kube-api-access-vjdbq\") pod \"877427df-588c-42f1-939a-988655d54a86\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " Jul 06 22:49:56.634548 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634479 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-service-ca\") pod \"877427df-588c-42f1-939a-988655d54a86\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " Jul 06 22:49:56.634548 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634526 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-serving-cert\") pod \"877427df-588c-42f1-939a-988655d54a86\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " Jul 06 22:49:56.634719 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634557 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-oauth-config\") pod \"877427df-588c-42f1-939a-988655d54a86\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " Jul 06 22:49:56.634719 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634623 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-oauth-serving-cert\") pod \"877427df-588c-42f1-939a-988655d54a86\" (UID: \"877427df-588c-42f1-939a-988655d54a86\") " Jul 06 22:49:56.634719 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634630 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-console-config" (OuterVolumeSpecName: "console-config") pod "877427df-588c-42f1-939a-988655d54a86" (UID: "877427df-588c-42f1-939a-988655d54a86"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:56.634719 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634665 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "877427df-588c-42f1-939a-988655d54a86" (UID: "877427df-588c-42f1-939a-988655d54a86"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:56.634912 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634897 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:49:56.634978 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634916 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:49:56.635044 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.634983 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-service-ca" (OuterVolumeSpecName: "service-ca") pod "877427df-588c-42f1-939a-988655d54a86" (UID: "877427df-588c-42f1-939a-988655d54a86"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:56.635141 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.635105 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "877427df-588c-42f1-939a-988655d54a86" (UID: "877427df-588c-42f1-939a-988655d54a86"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:56.636604 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.636575 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877427df-588c-42f1-939a-988655d54a86-kube-api-access-vjdbq" (OuterVolumeSpecName: "kube-api-access-vjdbq") pod "877427df-588c-42f1-939a-988655d54a86" (UID: "877427df-588c-42f1-939a-988655d54a86"). InnerVolumeSpecName "kube-api-access-vjdbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:49:56.636741 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.636630 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "877427df-588c-42f1-939a-988655d54a86" (UID: "877427df-588c-42f1-939a-988655d54a86"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:49:56.636741 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.636670 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "877427df-588c-42f1-939a-988655d54a86" (UID: "877427df-588c-42f1-939a-988655d54a86"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:49:56.736095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.736039 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjdbq\" (UniqueName: \"kubernetes.io/projected/877427df-588c-42f1-939a-988655d54a86-kube-api-access-vjdbq\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:49:56.736095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.736085 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:49:56.736095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.736097 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:49:56.736095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.736107 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/877427df-588c-42f1-939a-988655d54a86-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:49:56.736095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:56.736115 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/877427df-588c-42f1-939a-988655d54a86-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:49:57.277055 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.277021 2561 generic.go:356] "Generic (PLEG): container finished" podID="877427df-588c-42f1-939a-988655d54a86" containerID="9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506" exitCode=0 Jul 06 22:49:57.277463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.277061 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f56f85544-hbjqd" Jul 06 22:49:57.277463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.277083 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f56f85544-hbjqd" event={"ID":"877427df-588c-42f1-939a-988655d54a86","Type":"ContainerDied","Data":"9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506"} Jul 06 22:49:57.277463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.277123 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f56f85544-hbjqd" event={"ID":"877427df-588c-42f1-939a-988655d54a86","Type":"ContainerDied","Data":"f9d2e122c7c867fe06bc0fd162dad82e9e0d8a08b503db955c1b4a823530d1bb"} Jul 06 22:49:57.277463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.277142 2561 scope.go:117] "RemoveContainer" containerID="9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506" Jul 06 22:49:57.284847 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.284831 2561 scope.go:117] "RemoveContainer" containerID="9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506" Jul 06 22:49:57.285101 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:49:57.285080 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506\": container with ID starting with 9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506 not found: ID does not exist" containerID="9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506" Jul 06 22:49:57.285162 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.285110 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506"} err="failed to get container status \"9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506\": rpc error: code = NotFound desc = could not find container \"9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506\": container with ID starting with 9c4d8fbf6e402760d3a8af2c0e8f73e00d697b3e5b9b61e4838977f578e23506 not found: ID does not exist" Jul 06 22:49:57.291769 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.291743 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f56f85544-hbjqd"] Jul 06 22:49:57.295012 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:57.294987 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f56f85544-hbjqd"] Jul 06 22:49:59.221999 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:49:59.221455 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877427df-588c-42f1-939a-988655d54a86" path="/var/lib/kubelet/pods/877427df-588c-42f1-939a-988655d54a86/volumes" Jul 06 22:50:42.433978 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.433934 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cb959d9b-6qdwz"] Jul 06 22:50:42.434502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.434225 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="877427df-588c-42f1-939a-988655d54a86" containerName="console" Jul 06 22:50:42.434502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.434241 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="877427df-588c-42f1-939a-988655d54a86" containerName="console" Jul 06 22:50:42.434502 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.434304 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="877427df-588c-42f1-939a-988655d54a86" containerName="console" Jul 06 22:50:42.438415 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.438382 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.449581 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.449549 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cb959d9b-6qdwz"] Jul 06 22:50:42.588393 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.588355 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-service-ca\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.588570 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.588414 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-serving-cert\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.588570 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.588473 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-oauth-config\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.588570 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.588516 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-oauth-serving-cert\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.588570 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.588564 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-trusted-ca-bundle\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.588793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.588582 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtsqv\" (UniqueName: \"kubernetes.io/projected/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-kube-api-access-dtsqv\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.588793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.588633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-config\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.689273 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.689158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-service-ca\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.689273 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.689232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-serving-cert\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.689273 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.689258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-oauth-config\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.689563 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.689291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-oauth-serving-cert\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.689563 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.689344 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-trusted-ca-bundle\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.689563 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.689371 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsqv\" (UniqueName: \"kubernetes.io/projected/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-kube-api-access-dtsqv\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.689563 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.689409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-config\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.690104 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.690079 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-oauth-serving-cert\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.690104 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.690091 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-service-ca\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.690226 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.690082 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-config\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.690486 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.690450 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-trusted-ca-bundle\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.691786 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.691764 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-oauth-config\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.691979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.691961 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-serving-cert\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.697199 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.697173 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsqv\" (UniqueName: \"kubernetes.io/projected/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-kube-api-access-dtsqv\") pod \"console-6cb959d9b-6qdwz\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.748229 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.748181 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:42.883041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:42.883008 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cb959d9b-6qdwz"] Jul 06 22:50:42.886289 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:50:42.886245 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d2a0ee_ac17_4bfd_98ed_6ee78db2e0ef.slice/crio-6806914a13b2f445fbb8ebf2704f608c16e0dcb4c7a7a670f265f908874aadcc WatchSource:0}: Error finding container 6806914a13b2f445fbb8ebf2704f608c16e0dcb4c7a7a670f265f908874aadcc: Status 404 returned error can't find the container with id 6806914a13b2f445fbb8ebf2704f608c16e0dcb4c7a7a670f265f908874aadcc Jul 06 22:50:43.446542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:43.446498 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cb959d9b-6qdwz" event={"ID":"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef","Type":"ContainerStarted","Data":"732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73"} Jul 06 22:50:43.446964 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:43.446557 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cb959d9b-6qdwz" event={"ID":"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef","Type":"ContainerStarted","Data":"6806914a13b2f445fbb8ebf2704f608c16e0dcb4c7a7a670f265f908874aadcc"} Jul 06 22:50:43.463400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:43.463314 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cb959d9b-6qdwz" podStartSLOduration=1.463295954 podStartE2EDuration="1.463295954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:50:43.463092981 +0000 UTC m=+256.762794620" watchObservedRunningTime="2026-07-06 22:50:43.463295954 +0000 UTC m=+256.762997560" Jul 06 22:50:52.748749 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:52.748613 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:52.748749 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:52.748713 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:52.753743 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:52.753712 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:53.487503 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:53.487472 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:50:53.531349 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:50:53.531314 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696c85d7db-kc8vn"] Jul 06 22:51:18.557334 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.557264 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-696c85d7db-kc8vn" podUID="b46b9d10-5f7c-4436-84ab-4e15db16ce37" containerName="console" containerID="cri-o://a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931" gracePeriod=15 Jul 06 22:51:18.801066 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.801033 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:51:18.872900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.872794 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-service-ca\") pod \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " Jul 06 22:51:18.872900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.872843 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-trusted-ca-bundle\") pod \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " Jul 06 22:51:18.872900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.872863 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-oauth-serving-cert\") pod \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " Jul 06 22:51:18.872900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.872892 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6b66\" (UniqueName: \"kubernetes.io/projected/b46b9d10-5f7c-4436-84ab-4e15db16ce37-kube-api-access-j6b66\") pod \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " Jul 06 22:51:18.873207 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873026 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-config\") pod \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " Jul 06 22:51:18.873207 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873127 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-serving-cert\") pod \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " Jul 06 22:51:18.873207 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873160 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-oauth-config\") pod \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\" (UID: \"b46b9d10-5f7c-4436-84ab-4e15db16ce37\") " Jul 06 22:51:18.873363 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873299 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-service-ca" (OuterVolumeSpecName: "service-ca") pod "b46b9d10-5f7c-4436-84ab-4e15db16ce37" (UID: "b46b9d10-5f7c-4436-84ab-4e15db16ce37"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:51:18.873363 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873348 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b46b9d10-5f7c-4436-84ab-4e15db16ce37" (UID: "b46b9d10-5f7c-4436-84ab-4e15db16ce37"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:51:18.873468 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873364 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b46b9d10-5f7c-4436-84ab-4e15db16ce37" (UID: "b46b9d10-5f7c-4436-84ab-4e15db16ce37"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:51:18.873468 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873406 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-config" (OuterVolumeSpecName: "console-config") pod "b46b9d10-5f7c-4436-84ab-4e15db16ce37" (UID: "b46b9d10-5f7c-4436-84ab-4e15db16ce37"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:51:18.873468 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873458 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:18.873569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873471 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:18.873569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873480 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:18.873569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.873493 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:18.875157 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.875110 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b46b9d10-5f7c-4436-84ab-4e15db16ce37" (UID: "b46b9d10-5f7c-4436-84ab-4e15db16ce37"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:51:18.875266 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.875157 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b46b9d10-5f7c-4436-84ab-4e15db16ce37" (UID: "b46b9d10-5f7c-4436-84ab-4e15db16ce37"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:51:18.875266 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.875224 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46b9d10-5f7c-4436-84ab-4e15db16ce37-kube-api-access-j6b66" (OuterVolumeSpecName: "kube-api-access-j6b66") pod "b46b9d10-5f7c-4436-84ab-4e15db16ce37" (UID: "b46b9d10-5f7c-4436-84ab-4e15db16ce37"). InnerVolumeSpecName "kube-api-access-j6b66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:51:18.974255 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.974218 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:18.974255 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.974250 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b46b9d10-5f7c-4436-84ab-4e15db16ce37-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:18.974255 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:18.974260 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6b66\" (UniqueName: \"kubernetes.io/projected/b46b9d10-5f7c-4436-84ab-4e15db16ce37-kube-api-access-j6b66\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:19.580901 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.580808 2561 generic.go:356] "Generic (PLEG): container finished" podID="b46b9d10-5f7c-4436-84ab-4e15db16ce37" containerID="a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931" exitCode=0 Jul 06 22:51:19.580901 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.580858 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696c85d7db-kc8vn" Jul 06 22:51:19.580901 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.580879 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c85d7db-kc8vn" event={"ID":"b46b9d10-5f7c-4436-84ab-4e15db16ce37","Type":"ContainerDied","Data":"a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931"} Jul 06 22:51:19.581412 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.580927 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696c85d7db-kc8vn" event={"ID":"b46b9d10-5f7c-4436-84ab-4e15db16ce37","Type":"ContainerDied","Data":"b5897fe61d03c2cb2af65a4c4a94ce59449aafa47f11924ce2489e52b70609d1"} Jul 06 22:51:19.581412 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.580953 2561 scope.go:117] "RemoveContainer" containerID="a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931" Jul 06 22:51:19.589118 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.589098 2561 scope.go:117] "RemoveContainer" containerID="a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931" Jul 06 22:51:19.589458 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:51:19.589434 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931\": container with ID starting with a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931 not found: ID does not exist" containerID="a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931" Jul 06 22:51:19.589532 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.589467 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931"} err="failed to get container status \"a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931\": rpc error: code = NotFound desc = could not find container \"a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931\": container with ID starting with a61eb42836c557fd95319c92a40a19bed77f1a099c6696919f6a86ba0f167931 not found: ID does not exist" Jul 06 22:51:19.596187 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.596161 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-696c85d7db-kc8vn"] Jul 06 22:51:19.597889 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:19.597866 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-696c85d7db-kc8vn"] Jul 06 22:51:21.219662 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:21.219627 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b46b9d10-5f7c-4436-84ab-4e15db16ce37" path="/var/lib/kubelet/pods/b46b9d10-5f7c-4436-84ab-4e15db16ce37/volumes" Jul 06 22:51:25.175013 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.174974 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp"] Jul 06 22:51:25.175496 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.175254 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46b9d10-5f7c-4436-84ab-4e15db16ce37" containerName="console" Jul 06 22:51:25.175496 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.175266 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46b9d10-5f7c-4436-84ab-4e15db16ce37" containerName="console" Jul 06 22:51:25.175496 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.175318 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b46b9d10-5f7c-4436-84ab-4e15db16ce37" containerName="console" Jul 06 22:51:25.179757 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.179734 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.182388 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.182362 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 22:51:25.183311 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.183292 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 22:51:25.183387 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.183294 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h97th\"" Jul 06 22:51:25.186974 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.186925 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp"] Jul 06 22:51:25.224015 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.223961 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9vn2\" (UniqueName: \"kubernetes.io/projected/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-kube-api-access-q9vn2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.224015 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.224012 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.224230 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.224088 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.325261 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.325217 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9vn2\" (UniqueName: \"kubernetes.io/projected/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-kube-api-access-q9vn2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.325426 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.325267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.325426 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.325317 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.325747 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.325678 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.325799 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.325755 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.333754 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.333720 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9vn2\" (UniqueName: \"kubernetes.io/projected/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-kube-api-access-q9vn2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.490283 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.490191 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:25.624324 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:25.624290 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp"] Jul 06 22:51:25.627292 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:51:25.627260 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e2edc3_a226_4ed1_909c_ad2dbaadc573.slice/crio-239ddc79af205fc36f8f0c2ff68eb179c7e4c01de49530dd799e4f5aced3b3a9 WatchSource:0}: Error finding container 239ddc79af205fc36f8f0c2ff68eb179c7e4c01de49530dd799e4f5aced3b3a9: Status 404 returned error can't find the container with id 239ddc79af205fc36f8f0c2ff68eb179c7e4c01de49530dd799e4f5aced3b3a9 Jul 06 22:51:26.609808 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:26.609766 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" event={"ID":"f8e2edc3-a226-4ed1-909c-ad2dbaadc573","Type":"ContainerStarted","Data":"239ddc79af205fc36f8f0c2ff68eb179c7e4c01de49530dd799e4f5aced3b3a9"} Jul 06 22:51:27.090517 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:27.090482 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 22:51:27.095892 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:27.095868 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 22:51:27.097285 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:27.097263 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:51:27.103207 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:27.103184 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 22:51:27.103816 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:27.103790 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:51:31.635431 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:31.635330 2561 generic.go:356] "Generic (PLEG): container finished" podID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerID="a139193b2bc411145b479ea074ae1c2df0a00e7f461a206eb8b59ecd4bfa10c0" exitCode=0 Jul 06 22:51:31.635431 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:31.635404 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" event={"ID":"f8e2edc3-a226-4ed1-909c-ad2dbaadc573","Type":"ContainerDied","Data":"a139193b2bc411145b479ea074ae1c2df0a00e7f461a206eb8b59ecd4bfa10c0"} Jul 06 22:51:31.636439 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:31.636422 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:51:33.646084 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:33.645989 2561 generic.go:356] "Generic (PLEG): container finished" podID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerID="8753260ee1a24164804594700806cf39a0f604a132dad937e8cb57f6d91c2819" exitCode=0 Jul 06 22:51:33.646084 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:33.646051 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" event={"ID":"f8e2edc3-a226-4ed1-909c-ad2dbaadc573","Type":"ContainerDied","Data":"8753260ee1a24164804594700806cf39a0f604a132dad937e8cb57f6d91c2819"} Jul 06 22:51:40.679553 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:40.679514 2561 generic.go:356] "Generic (PLEG): container finished" podID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerID="11ce697ec13ece4a7abd6cc677e6273752eed08636fdef5a921c614e87b5e579" exitCode=0 Jul 06 22:51:40.679996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:40.679591 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" event={"ID":"f8e2edc3-a226-4ed1-909c-ad2dbaadc573","Type":"ContainerDied","Data":"11ce697ec13ece4a7abd6cc677e6273752eed08636fdef5a921c614e87b5e579"} Jul 06 22:51:41.809136 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.809104 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:41.872095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.872054 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-bundle\") pod \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " Jul 06 22:51:41.872271 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.872118 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-util\") pod \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " Jul 06 22:51:41.872271 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.872165 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9vn2\" (UniqueName: \"kubernetes.io/projected/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-kube-api-access-q9vn2\") pod \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\" (UID: \"f8e2edc3-a226-4ed1-909c-ad2dbaadc573\") " Jul 06 22:51:41.872954 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.872922 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-bundle" (OuterVolumeSpecName: "bundle") pod "f8e2edc3-a226-4ed1-909c-ad2dbaadc573" (UID: "f8e2edc3-a226-4ed1-909c-ad2dbaadc573"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:51:41.874419 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.874388 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-kube-api-access-q9vn2" (OuterVolumeSpecName: "kube-api-access-q9vn2") pod "f8e2edc3-a226-4ed1-909c-ad2dbaadc573" (UID: "f8e2edc3-a226-4ed1-909c-ad2dbaadc573"). InnerVolumeSpecName "kube-api-access-q9vn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:51:41.877125 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.877096 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-util" (OuterVolumeSpecName: "util") pod "f8e2edc3-a226-4ed1-909c-ad2dbaadc573" (UID: "f8e2edc3-a226-4ed1-909c-ad2dbaadc573"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:51:41.973038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.972955 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:41.973038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.972990 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:41.973038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:41.972999 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9vn2\" (UniqueName: \"kubernetes.io/projected/f8e2edc3-a226-4ed1-909c-ad2dbaadc573-kube-api-access-q9vn2\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:42.688991 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:42.688951 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" event={"ID":"f8e2edc3-a226-4ed1-909c-ad2dbaadc573","Type":"ContainerDied","Data":"239ddc79af205fc36f8f0c2ff68eb179c7e4c01de49530dd799e4f5aced3b3a9"} Jul 06 22:51:42.688991 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:42.688987 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754092mqp" Jul 06 22:51:42.689197 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:42.689006 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="239ddc79af205fc36f8f0c2ff68eb179c7e4c01de49530dd799e4f5aced3b3a9" Jul 06 22:51:48.072042 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072003 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn"] Jul 06 22:51:48.072440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072273 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerName="pull" Jul 06 22:51:48.072440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072284 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerName="pull" Jul 06 22:51:48.072440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072297 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerName="extract" Jul 06 22:51:48.072440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072303 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerName="extract" Jul 06 22:51:48.072440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072310 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerName="util" Jul 06 22:51:48.072440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072316 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerName="util" Jul 06 22:51:48.072440 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.072365 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8e2edc3-a226-4ed1-909c-ad2dbaadc573" containerName="extract" Jul 06 22:51:48.126911 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.126877 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn"] Jul 06 22:51:48.127087 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.126992 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.129836 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.129802 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cjv6l\"" Jul 06 22:51:48.129970 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.129805 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:51:48.129970 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.129814 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 22:51:48.226652 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.226608 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/573511ca-5af2-40c7-910d-d074a6319848-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sqbfn\" (UID: \"573511ca-5af2-40c7-910d-d074a6319848\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.226877 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.226760 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz2fb\" (UniqueName: \"kubernetes.io/projected/573511ca-5af2-40c7-910d-d074a6319848-kube-api-access-hz2fb\") pod \"cert-manager-operator-controller-manager-55c444bf86-sqbfn\" (UID: \"573511ca-5af2-40c7-910d-d074a6319848\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.327681 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.327577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/573511ca-5af2-40c7-910d-d074a6319848-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sqbfn\" (UID: \"573511ca-5af2-40c7-910d-d074a6319848\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.327681 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.327651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz2fb\" (UniqueName: \"kubernetes.io/projected/573511ca-5af2-40c7-910d-d074a6319848-kube-api-access-hz2fb\") pod \"cert-manager-operator-controller-manager-55c444bf86-sqbfn\" (UID: \"573511ca-5af2-40c7-910d-d074a6319848\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.328052 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.328025 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/573511ca-5af2-40c7-910d-d074a6319848-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sqbfn\" (UID: \"573511ca-5af2-40c7-910d-d074a6319848\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.336909 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.336876 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz2fb\" (UniqueName: \"kubernetes.io/projected/573511ca-5af2-40c7-910d-d074a6319848-kube-api-access-hz2fb\") pod \"cert-manager-operator-controller-manager-55c444bf86-sqbfn\" (UID: \"573511ca-5af2-40c7-910d-d074a6319848\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.436479 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.436436 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" Jul 06 22:51:48.599622 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.599536 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn"] Jul 06 22:51:48.603311 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:51:48.603282 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573511ca_5af2_40c7_910d_d074a6319848.slice/crio-e608145082931dac204e54f9b6fad6336c55cfce56e9a7b2dd9c9640920b8573 WatchSource:0}: Error finding container e608145082931dac204e54f9b6fad6336c55cfce56e9a7b2dd9c9640920b8573: Status 404 returned error can't find the container with id e608145082931dac204e54f9b6fad6336c55cfce56e9a7b2dd9c9640920b8573 Jul 06 22:51:48.712750 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:48.712712 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" event={"ID":"573511ca-5af2-40c7-910d-d074a6319848","Type":"ContainerStarted","Data":"e608145082931dac204e54f9b6fad6336c55cfce56e9a7b2dd9c9640920b8573"} Jul 06 22:51:51.728384 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:51.728344 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" event={"ID":"573511ca-5af2-40c7-910d-d074a6319848","Type":"ContainerStarted","Data":"d33f70c8420e0e8305c8f581dcc828042042869305168ce6905b28e0a58e7918"} Jul 06 22:51:51.751083 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:51.751026 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sqbfn" podStartSLOduration=1.62357152 podStartE2EDuration="3.751009563s" totalImagesPullingTime="2.127438043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:51:51.748409676 +0000 UTC m=+325.048111299" watchObservedRunningTime="2026-07-06 22:51:51.751009563 +0000 UTC m=+325.050711165" Jul 06 22:51:52.563653 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.563615 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6"] Jul 06 22:51:52.566111 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.566088 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.570900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.570875 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 22:51:52.571038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.570876 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h97th\"" Jul 06 22:51:52.571647 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.571628 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 22:51:52.582904 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.582874 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6"] Jul 06 22:51:52.664755 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.664712 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.664755 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.664762 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjb2\" (UniqueName: \"kubernetes.io/projected/d7915521-4c86-4273-ab2d-32f3f97263f9-kube-api-access-kwjb2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.664988 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.664843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.765436 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.765389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.765436 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.765439 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjb2\" (UniqueName: \"kubernetes.io/projected/d7915521-4c86-4273-ab2d-32f3f97263f9-kube-api-access-kwjb2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.765997 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.765470 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.765997 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.765805 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.765997 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.765845 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.775545 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.775510 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjb2\" (UniqueName: \"kubernetes.io/projected/d7915521-4c86-4273-ab2d-32f3f97263f9-kube-api-access-kwjb2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:52.874891 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:52.874797 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:53.009401 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:53.009370 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6"] Jul 06 22:51:53.011768 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:51:53.011739 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7915521_4c86_4273_ab2d_32f3f97263f9.slice/crio-e40422562404bac91f125859e649143de1d14a6bf947f0c02fde48c3d057d2e3 WatchSource:0}: Error finding container e40422562404bac91f125859e649143de1d14a6bf947f0c02fde48c3d057d2e3: Status 404 returned error can't find the container with id e40422562404bac91f125859e649143de1d14a6bf947f0c02fde48c3d057d2e3 Jul 06 22:51:53.737447 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:53.737411 2561 generic.go:356] "Generic (PLEG): container finished" podID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerID="8454692a22b877e98c1f27617981c2435025c42477e7bed5c0ef6490681ab7a5" exitCode=0 Jul 06 22:51:53.737635 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:53.737480 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" event={"ID":"d7915521-4c86-4273-ab2d-32f3f97263f9","Type":"ContainerDied","Data":"8454692a22b877e98c1f27617981c2435025c42477e7bed5c0ef6490681ab7a5"} Jul 06 22:51:53.737635 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:53.737524 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" event={"ID":"d7915521-4c86-4273-ab2d-32f3f97263f9","Type":"ContainerStarted","Data":"e40422562404bac91f125859e649143de1d14a6bf947f0c02fde48c3d057d2e3"} Jul 06 22:51:56.752244 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:56.752206 2561 generic.go:356] "Generic (PLEG): container finished" podID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerID="71ebde3f7d4c8a197544cea57218a61afe7978b2f9b9da05bf4680b31f4226a7" exitCode=0 Jul 06 22:51:56.752663 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:56.752299 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" event={"ID":"d7915521-4c86-4273-ab2d-32f3f97263f9","Type":"ContainerDied","Data":"71ebde3f7d4c8a197544cea57218a61afe7978b2f9b9da05bf4680b31f4226a7"} Jul 06 22:51:57.758885 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:57.758840 2561 generic.go:356] "Generic (PLEG): container finished" podID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerID="b9446eefddab7fbfea7ecf16fe03106f2037a0a8b93c4802a3b84b7cbe76a719" exitCode=0 Jul 06 22:51:57.759350 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:57.758901 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" event={"ID":"d7915521-4c86-4273-ab2d-32f3f97263f9","Type":"ContainerDied","Data":"b9446eefddab7fbfea7ecf16fe03106f2037a0a8b93c4802a3b84b7cbe76a719"} Jul 06 22:51:58.900028 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:58.900000 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:59.015372 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.015272 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-util\") pod \"d7915521-4c86-4273-ab2d-32f3f97263f9\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " Jul 06 22:51:59.015372 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.015330 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwjb2\" (UniqueName: \"kubernetes.io/projected/d7915521-4c86-4273-ab2d-32f3f97263f9-kube-api-access-kwjb2\") pod \"d7915521-4c86-4273-ab2d-32f3f97263f9\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " Jul 06 22:51:59.015594 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.015400 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-bundle\") pod \"d7915521-4c86-4273-ab2d-32f3f97263f9\" (UID: \"d7915521-4c86-4273-ab2d-32f3f97263f9\") " Jul 06 22:51:59.015844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.015815 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-bundle" (OuterVolumeSpecName: "bundle") pod "d7915521-4c86-4273-ab2d-32f3f97263f9" (UID: "d7915521-4c86-4273-ab2d-32f3f97263f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:51:59.017458 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.017435 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7915521-4c86-4273-ab2d-32f3f97263f9-kube-api-access-kwjb2" (OuterVolumeSpecName: "kube-api-access-kwjb2") pod "d7915521-4c86-4273-ab2d-32f3f97263f9" (UID: "d7915521-4c86-4273-ab2d-32f3f97263f9"). InnerVolumeSpecName "kube-api-access-kwjb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:51:59.019518 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.019486 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-util" (OuterVolumeSpecName: "util") pod "d7915521-4c86-4273-ab2d-32f3f97263f9" (UID: "d7915521-4c86-4273-ab2d-32f3f97263f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:51:59.116749 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.116677 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:59.116749 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.116739 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwjb2\" (UniqueName: \"kubernetes.io/projected/d7915521-4c86-4273-ab2d-32f3f97263f9-kube-api-access-kwjb2\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:59.116749 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.116750 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7915521-4c86-4273-ab2d-32f3f97263f9-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:51:59.769873 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.769840 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" Jul 06 22:51:59.769873 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.769860 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7jbs6" event={"ID":"d7915521-4c86-4273-ab2d-32f3f97263f9","Type":"ContainerDied","Data":"e40422562404bac91f125859e649143de1d14a6bf947f0c02fde48c3d057d2e3"} Jul 06 22:51:59.770077 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:51:59.769908 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e40422562404bac91f125859e649143de1d14a6bf947f0c02fde48c3d057d2e3" Jul 06 22:52:04.367610 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367575 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cmq5j"] Jul 06 22:52:04.368038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367858 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerName="pull" Jul 06 22:52:04.368038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367870 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerName="pull" Jul 06 22:52:04.368038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367881 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerName="extract" Jul 06 22:52:04.368038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367886 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerName="extract" Jul 06 22:52:04.368038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367897 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerName="util" Jul 06 22:52:04.368038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367902 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerName="util" Jul 06 22:52:04.368038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.367945 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7915521-4c86-4273-ab2d-32f3f97263f9" containerName="extract" Jul 06 22:52:04.371531 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.371510 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.374593 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.374568 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qnnlm\"" Jul 06 22:52:04.374755 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.374606 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 22:52:04.374755 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.374606 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 22:52:04.380198 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.380172 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cmq5j"] Jul 06 22:52:04.563121 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.563064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4qwl\" (UniqueName: \"kubernetes.io/projected/0897be36-d964-469f-a3b6-5a1978bc4e9e-kube-api-access-w4qwl\") pod \"cert-manager-55b4d954d5-cmq5j\" (UID: \"0897be36-d964-469f-a3b6-5a1978bc4e9e\") " pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.563308 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.563187 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0897be36-d964-469f-a3b6-5a1978bc4e9e-bound-sa-token\") pod \"cert-manager-55b4d954d5-cmq5j\" (UID: \"0897be36-d964-469f-a3b6-5a1978bc4e9e\") " pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.664193 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.664082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4qwl\" (UniqueName: \"kubernetes.io/projected/0897be36-d964-469f-a3b6-5a1978bc4e9e-kube-api-access-w4qwl\") pod \"cert-manager-55b4d954d5-cmq5j\" (UID: \"0897be36-d964-469f-a3b6-5a1978bc4e9e\") " pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.664193 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.664155 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0897be36-d964-469f-a3b6-5a1978bc4e9e-bound-sa-token\") pod \"cert-manager-55b4d954d5-cmq5j\" (UID: \"0897be36-d964-469f-a3b6-5a1978bc4e9e\") " pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.672788 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.672757 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0897be36-d964-469f-a3b6-5a1978bc4e9e-bound-sa-token\") pod \"cert-manager-55b4d954d5-cmq5j\" (UID: \"0897be36-d964-469f-a3b6-5a1978bc4e9e\") " pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.672979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.672956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4qwl\" (UniqueName: \"kubernetes.io/projected/0897be36-d964-469f-a3b6-5a1978bc4e9e-kube-api-access-w4qwl\") pod \"cert-manager-55b4d954d5-cmq5j\" (UID: \"0897be36-d964-469f-a3b6-5a1978bc4e9e\") " pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.681641 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.681612 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cmq5j" Jul 06 22:52:04.814950 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:04.814913 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cmq5j"] Jul 06 22:52:04.818088 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:52:04.818050 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0897be36_d964_469f_a3b6_5a1978bc4e9e.slice/crio-1bfa72feeb86867035a607f2b9240cd335bdf8a6695ca940ad5fc4b7b254487d WatchSource:0}: Error finding container 1bfa72feeb86867035a607f2b9240cd335bdf8a6695ca940ad5fc4b7b254487d: Status 404 returned error can't find the container with id 1bfa72feeb86867035a607f2b9240cd335bdf8a6695ca940ad5fc4b7b254487d Jul 06 22:52:05.800199 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:05.800158 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cmq5j" event={"ID":"0897be36-d964-469f-a3b6-5a1978bc4e9e","Type":"ContainerStarted","Data":"1bfa72feeb86867035a607f2b9240cd335bdf8a6695ca940ad5fc4b7b254487d"} Jul 06 22:52:07.810931 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:07.810889 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cmq5j" event={"ID":"0897be36-d964-469f-a3b6-5a1978bc4e9e","Type":"ContainerStarted","Data":"dbd8afadfc7784e0681f16efcb275b1dd57030f8640eee98457e32d6d22df48b"} Jul 06 22:52:07.827775 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:07.827717 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cmq5j" podStartSLOduration=1.312611654 podStartE2EDuration="3.827699457s" totalImagesPullingTime="2.515087803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:52:07.827044549 +0000 UTC m=+341.126746153" watchObservedRunningTime="2026-07-06 22:52:07.827699457 +0000 UTC m=+341.127401050" Jul 06 22:52:21.230412 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.230327 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm"] Jul 06 22:52:21.233581 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.233560 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.236272 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.236246 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h97th\"" Jul 06 22:52:21.236423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.236277 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 22:52:21.237284 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.237266 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 22:52:21.242955 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.242927 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm"] Jul 06 22:52:21.291348 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.291310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7n7q\" (UniqueName: \"kubernetes.io/projected/2e62d0e2-0761-4025-943b-017317093dd5-kube-api-access-x7n7q\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.291542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.291366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.291542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.291389 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.392658 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.392601 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.392658 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.392658 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.392887 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.392817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7n7q\" (UniqueName: \"kubernetes.io/projected/2e62d0e2-0761-4025-943b-017317093dd5-kube-api-access-x7n7q\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.393052 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.393030 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.393100 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.393082 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.401971 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.401929 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7n7q\" (UniqueName: \"kubernetes.io/projected/2e62d0e2-0761-4025-943b-017317093dd5-kube-api-access-x7n7q\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.543741 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.543613 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:21.678041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.678015 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm"] Jul 06 22:52:21.679864 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:52:21.679836 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e62d0e2_0761_4025_943b_017317093dd5.slice/crio-1f800fa140d4f9c9cd851949bd854648101dabc81c35aa9f99981a4a77a3d46e WatchSource:0}: Error finding container 1f800fa140d4f9c9cd851949bd854648101dabc81c35aa9f99981a4a77a3d46e: Status 404 returned error can't find the container with id 1f800fa140d4f9c9cd851949bd854648101dabc81c35aa9f99981a4a77a3d46e Jul 06 22:52:21.872592 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.872499 2561 generic.go:356] "Generic (PLEG): container finished" podID="2e62d0e2-0761-4025-943b-017317093dd5" containerID="dbc95331c6e8e52bfe651caeb15bb5bb702db61b0a64721a84158e942a8e8431" exitCode=0 Jul 06 22:52:21.872760 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.872589 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" event={"ID":"2e62d0e2-0761-4025-943b-017317093dd5","Type":"ContainerDied","Data":"dbc95331c6e8e52bfe651caeb15bb5bb702db61b0a64721a84158e942a8e8431"} Jul 06 22:52:21.872760 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:21.872645 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" event={"ID":"2e62d0e2-0761-4025-943b-017317093dd5","Type":"ContainerStarted","Data":"1f800fa140d4f9c9cd851949bd854648101dabc81c35aa9f99981a4a77a3d46e"} Jul 06 22:52:22.878479 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:22.878433 2561 generic.go:356] "Generic (PLEG): container finished" podID="2e62d0e2-0761-4025-943b-017317093dd5" containerID="7201afe308ed3e67180d015769534d10a5387c8380362af0067347a499d47f73" exitCode=0 Jul 06 22:52:22.878953 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:22.878502 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" event={"ID":"2e62d0e2-0761-4025-943b-017317093dd5","Type":"ContainerDied","Data":"7201afe308ed3e67180d015769534d10a5387c8380362af0067347a499d47f73"} Jul 06 22:52:23.885043 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:23.884994 2561 generic.go:356] "Generic (PLEG): container finished" podID="2e62d0e2-0761-4025-943b-017317093dd5" containerID="9a57ff22f2ec3cd5fd6b0dcf02507f45536b848893414eb28c72e0747f02722f" exitCode=0 Jul 06 22:52:23.885043 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:23.885043 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" event={"ID":"2e62d0e2-0761-4025-943b-017317093dd5","Type":"ContainerDied","Data":"9a57ff22f2ec3cd5fd6b0dcf02507f45536b848893414eb28c72e0747f02722f"} Jul 06 22:52:25.039470 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.039441 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:25.122651 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.122603 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-bundle\") pod \"2e62d0e2-0761-4025-943b-017317093dd5\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " Jul 06 22:52:25.122846 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.122707 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-util\") pod \"2e62d0e2-0761-4025-943b-017317093dd5\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " Jul 06 22:52:25.122846 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.122745 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7n7q\" (UniqueName: \"kubernetes.io/projected/2e62d0e2-0761-4025-943b-017317093dd5-kube-api-access-x7n7q\") pod \"2e62d0e2-0761-4025-943b-017317093dd5\" (UID: \"2e62d0e2-0761-4025-943b-017317093dd5\") " Jul 06 22:52:25.123428 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.123402 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-bundle" (OuterVolumeSpecName: "bundle") pod "2e62d0e2-0761-4025-943b-017317093dd5" (UID: "2e62d0e2-0761-4025-943b-017317093dd5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:52:25.124850 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.124820 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e62d0e2-0761-4025-943b-017317093dd5-kube-api-access-x7n7q" (OuterVolumeSpecName: "kube-api-access-x7n7q") pod "2e62d0e2-0761-4025-943b-017317093dd5" (UID: "2e62d0e2-0761-4025-943b-017317093dd5"). InnerVolumeSpecName "kube-api-access-x7n7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:52:25.128422 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.128397 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-util" (OuterVolumeSpecName: "util") pod "2e62d0e2-0761-4025-943b-017317093dd5" (UID: "2e62d0e2-0761-4025-943b-017317093dd5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:52:25.223357 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.223327 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:52:25.223357 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.223355 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e62d0e2-0761-4025-943b-017317093dd5-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:52:25.223628 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.223369 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7n7q\" (UniqueName: \"kubernetes.io/projected/2e62d0e2-0761-4025-943b-017317093dd5-kube-api-access-x7n7q\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:52:25.894856 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.894823 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" Jul 06 22:52:25.895041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.894857 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af75vsm" event={"ID":"2e62d0e2-0761-4025-943b-017317093dd5","Type":"ContainerDied","Data":"1f800fa140d4f9c9cd851949bd854648101dabc81c35aa9f99981a4a77a3d46e"} Jul 06 22:52:25.895041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:25.894902 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f800fa140d4f9c9cd851949bd854648101dabc81c35aa9f99981a4a77a3d46e" Jul 06 22:52:36.325795 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.325753 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2"] Jul 06 22:52:36.326214 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.326019 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e62d0e2-0761-4025-943b-017317093dd5" containerName="extract" Jul 06 22:52:36.326214 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.326030 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e62d0e2-0761-4025-943b-017317093dd5" containerName="extract" Jul 06 22:52:36.326214 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.326047 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e62d0e2-0761-4025-943b-017317093dd5" containerName="util" Jul 06 22:52:36.326214 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.326053 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e62d0e2-0761-4025-943b-017317093dd5" containerName="util" Jul 06 22:52:36.326214 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.326064 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e62d0e2-0761-4025-943b-017317093dd5" containerName="pull" Jul 06 22:52:36.326214 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.326069 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e62d0e2-0761-4025-943b-017317093dd5" containerName="pull" Jul 06 22:52:36.326214 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.326127 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e62d0e2-0761-4025-943b-017317093dd5" containerName="extract" Jul 06 22:52:36.333838 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.333804 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.337163 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.337133 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 22:52:36.337460 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.337442 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 22:52:36.337569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.337505 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 22:52:36.337629 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.337572 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 22:52:36.337629 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.337607 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dx78r\"" Jul 06 22:52:36.350370 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.350338 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2"] Jul 06 22:52:36.421235 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.421187 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c93e238-ab31-4458-8a69-b797e29aa41b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.421441 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.421244 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c93e238-ab31-4458-8a69-b797e29aa41b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.421441 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.421270 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c93e238-ab31-4458-8a69-b797e29aa41b-tmp\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.421441 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.421312 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jt9p\" (UniqueName: \"kubernetes.io/projected/6c93e238-ab31-4458-8a69-b797e29aa41b-kube-api-access-2jt9p\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.421441 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.421339 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c93e238-ab31-4458-8a69-b797e29aa41b-manifests\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.522121 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.522079 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c93e238-ab31-4458-8a69-b797e29aa41b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.522314 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.522132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c93e238-ab31-4458-8a69-b797e29aa41b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.522383 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.522304 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c93e238-ab31-4458-8a69-b797e29aa41b-tmp\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.522383 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.522366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jt9p\" (UniqueName: \"kubernetes.io/projected/6c93e238-ab31-4458-8a69-b797e29aa41b-kube-api-access-2jt9p\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.522484 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.522402 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c93e238-ab31-4458-8a69-b797e29aa41b-manifests\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.522849 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.522817 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6c93e238-ab31-4458-8a69-b797e29aa41b-manifests\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.524549 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.524525 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c93e238-ab31-4458-8a69-b797e29aa41b-tmp\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.524669 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.524622 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6c93e238-ab31-4458-8a69-b797e29aa41b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.524809 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.524674 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c93e238-ab31-4458-8a69-b797e29aa41b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.532210 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.532184 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jt9p\" (UniqueName: \"kubernetes.io/projected/6c93e238-ab31-4458-8a69-b797e29aa41b-kube-api-access-2jt9p\") pod \"opendatahub-operator-controller-manager-84df5575d5-cs6d2\" (UID: \"6c93e238-ab31-4458-8a69-b797e29aa41b\") " pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.644922 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.644827 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:36.805349 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.805314 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2"] Jul 06 22:52:36.808850 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:52:36.808810 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c93e238_ab31_4458_8a69_b797e29aa41b.slice/crio-c9c7d770535f138bd54e3519ad5420af4c3c97d1c295c36fc162123b236756cb WatchSource:0}: Error finding container c9c7d770535f138bd54e3519ad5420af4c3c97d1c295c36fc162123b236756cb: Status 404 returned error can't find the container with id c9c7d770535f138bd54e3519ad5420af4c3c97d1c295c36fc162123b236756cb Jul 06 22:52:36.945423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:36.945381 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" event={"ID":"6c93e238-ab31-4458-8a69-b797e29aa41b","Type":"ContainerStarted","Data":"c9c7d770535f138bd54e3519ad5420af4c3c97d1c295c36fc162123b236756cb"} Jul 06 22:52:39.962012 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:39.961977 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" event={"ID":"6c93e238-ab31-4458-8a69-b797e29aa41b","Type":"ContainerStarted","Data":"2d15929cb532dd5ea95b1b0ff0e8288dc37059d8999ac9b07eb79d6985b60f6e"} Jul 06 22:52:41.971900 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:41.971862 2561 generic.go:356] "Generic (PLEG): container finished" podID="6c93e238-ab31-4458-8a69-b797e29aa41b" containerID="2d15929cb532dd5ea95b1b0ff0e8288dc37059d8999ac9b07eb79d6985b60f6e" exitCode=0 Jul 06 22:52:41.972311 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:41.971929 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" event={"ID":"6c93e238-ab31-4458-8a69-b797e29aa41b","Type":"ContainerDied","Data":"2d15929cb532dd5ea95b1b0ff0e8288dc37059d8999ac9b07eb79d6985b60f6e"} Jul 06 22:52:42.978047 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:42.978009 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" event={"ID":"6c93e238-ab31-4458-8a69-b797e29aa41b","Type":"ContainerStarted","Data":"467dbc29476f4d0e558e050716bbbdba1e12681ba0a7e013fd43cb27a26025d5"} Jul 06 22:52:42.978567 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:42.978244 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:43.017988 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:43.017935 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" podStartSLOduration=2.464319301 podStartE2EDuration="7.017917656s" totalImagesPullingTime="3.389025406s" totalInitContainerRuntime="1.164572949s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:52:43.017910696 +0000 UTC m=+376.317612297" watchObservedRunningTime="2026-07-06 22:52:43.017917656 +0000 UTC m=+376.317619259" Jul 06 22:52:53.984427 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:53.984393 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84df5575d5-cs6d2" Jul 06 22:52:56.622154 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.622119 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s"] Jul 06 22:52:56.625797 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.625771 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.630116 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.630091 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 22:52:56.630451 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.630436 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 22:52:56.631275 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.631254 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-snqg8\"" Jul 06 22:52:56.741159 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.741122 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s"] Jul 06 22:52:56.769859 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.769814 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wvqp8"] Jul 06 22:52:56.773501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.773481 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:56.776456 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.776421 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4hwtm\"" Jul 06 22:52:56.776632 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.776609 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 22:52:56.786605 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.786570 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wvqp8"] Jul 06 22:52:56.797929 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.797898 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20fa170-f482-45a3-ac02-391f1a736daf-tmp\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.798098 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.797960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20fa170-f482-45a3-ac02-391f1a736daf-tls-certs\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.798098 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.797983 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvfhj\" (UniqueName: \"kubernetes.io/projected/e20fa170-f482-45a3-ac02-391f1a736daf-kube-api-access-mvfhj\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.899029 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.898915 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r72rl\" (UniqueName: \"kubernetes.io/projected/650d898d-1738-4b98-b1df-7ce751e04ab8-kube-api-access-r72rl\") pod \"odh-model-controller-7998c67dc4-wvqp8\" (UID: \"650d898d-1738-4b98-b1df-7ce751e04ab8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:56.899029 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.898969 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/650d898d-1738-4b98-b1df-7ce751e04ab8-cert\") pod \"odh-model-controller-7998c67dc4-wvqp8\" (UID: \"650d898d-1738-4b98-b1df-7ce751e04ab8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:56.899029 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.899019 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20fa170-f482-45a3-ac02-391f1a736daf-tls-certs\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.899332 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.899054 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvfhj\" (UniqueName: \"kubernetes.io/projected/e20fa170-f482-45a3-ac02-391f1a736daf-kube-api-access-mvfhj\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.899332 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.899119 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20fa170-f482-45a3-ac02-391f1a736daf-tmp\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.901390 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.901352 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e20fa170-f482-45a3-ac02-391f1a736daf-tmp\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.901574 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.901555 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20fa170-f482-45a3-ac02-391f1a736daf-tls-certs\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.907870 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.907843 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvfhj\" (UniqueName: \"kubernetes.io/projected/e20fa170-f482-45a3-ac02-391f1a736daf-kube-api-access-mvfhj\") pod \"kube-auth-proxy-54f95589f8-jqc4s\" (UID: \"e20fa170-f482-45a3-ac02-391f1a736daf\") " pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:56.935348 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:56.935315 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" Jul 06 22:52:57.001117 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.000491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r72rl\" (UniqueName: \"kubernetes.io/projected/650d898d-1738-4b98-b1df-7ce751e04ab8-kube-api-access-r72rl\") pod \"odh-model-controller-7998c67dc4-wvqp8\" (UID: \"650d898d-1738-4b98-b1df-7ce751e04ab8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:57.001117 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.000541 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/650d898d-1738-4b98-b1df-7ce751e04ab8-cert\") pod \"odh-model-controller-7998c67dc4-wvqp8\" (UID: \"650d898d-1738-4b98-b1df-7ce751e04ab8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:57.004025 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.003953 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/650d898d-1738-4b98-b1df-7ce751e04ab8-cert\") pod \"odh-model-controller-7998c67dc4-wvqp8\" (UID: \"650d898d-1738-4b98-b1df-7ce751e04ab8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:57.015212 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.015176 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r72rl\" (UniqueName: \"kubernetes.io/projected/650d898d-1738-4b98-b1df-7ce751e04ab8-kube-api-access-r72rl\") pod \"odh-model-controller-7998c67dc4-wvqp8\" (UID: \"650d898d-1738-4b98-b1df-7ce751e04ab8\") " pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:57.084159 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.084119 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:52:57.091699 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.091661 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s"] Jul 06 22:52:57.094509 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:52:57.094478 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20fa170_f482_45a3_ac02_391f1a736daf.slice/crio-a6f21454da2bd2b09d9067aef8f5bd8ac03e0dfdfd492ff33d5027479b175700 WatchSource:0}: Error finding container a6f21454da2bd2b09d9067aef8f5bd8ac03e0dfdfd492ff33d5027479b175700: Status 404 returned error can't find the container with id a6f21454da2bd2b09d9067aef8f5bd8ac03e0dfdfd492ff33d5027479b175700 Jul 06 22:52:57.256370 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.256336 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wvqp8"] Jul 06 22:52:57.259871 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:52:57.259841 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650d898d_1738_4b98_b1df_7ce751e04ab8.slice/crio-3a7e6d22caa7e7d8e75827e01ac55bc11baafa5095fbd359984e759c25a7157b WatchSource:0}: Error finding container 3a7e6d22caa7e7d8e75827e01ac55bc11baafa5095fbd359984e759c25a7157b: Status 404 returned error can't find the container with id 3a7e6d22caa7e7d8e75827e01ac55bc11baafa5095fbd359984e759c25a7157b Jul 06 22:52:57.690385 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.690348 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cbrc5"] Jul 06 22:52:57.693999 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.693964 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:57.696600 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.696570 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 22:52:57.696892 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.696870 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nwmmk\"" Jul 06 22:52:57.785862 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.785773 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cbrc5"] Jul 06 22:52:57.807521 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.807435 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l45t8\" (UniqueName: \"kubernetes.io/projected/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-kube-api-access-l45t8\") pod \"model-serving-api-54d89bd5fc-cbrc5\" (UID: \"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:57.807521 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.807488 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-tls-certs\") pod \"model-serving-api-54d89bd5fc-cbrc5\" (UID: \"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:57.908830 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.908795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l45t8\" (UniqueName: \"kubernetes.io/projected/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-kube-api-access-l45t8\") pod \"model-serving-api-54d89bd5fc-cbrc5\" (UID: \"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:57.909012 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.908839 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-tls-certs\") pod \"model-serving-api-54d89bd5fc-cbrc5\" (UID: \"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:57.909094 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:52:57.909074 2561 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 06 22:52:57.909164 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:52:57.909152 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-tls-certs podName:a1e21ab9-d7e9-4f0d-9a03-8699c11061b6 nodeName:}" failed. No retries permitted until 2026-07-06 22:52:58.40912249 +0000 UTC m=+391.708824086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-tls-certs") pod "model-serving-api-54d89bd5fc-cbrc5" (UID: "a1e21ab9-d7e9-4f0d-9a03-8699c11061b6") : secret "model-serving-api-tls" not found Jul 06 22:52:57.930817 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:57.930783 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l45t8\" (UniqueName: \"kubernetes.io/projected/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-kube-api-access-l45t8\") pod \"model-serving-api-54d89bd5fc-cbrc5\" (UID: \"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:58.049478 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:58.049383 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" event={"ID":"e20fa170-f482-45a3-ac02-391f1a736daf","Type":"ContainerStarted","Data":"a6f21454da2bd2b09d9067aef8f5bd8ac03e0dfdfd492ff33d5027479b175700"} Jul 06 22:52:58.053138 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:58.053101 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" event={"ID":"650d898d-1738-4b98-b1df-7ce751e04ab8","Type":"ContainerStarted","Data":"3a7e6d22caa7e7d8e75827e01ac55bc11baafa5095fbd359984e759c25a7157b"} Jul 06 22:52:58.414315 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:58.414214 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-tls-certs\") pod \"model-serving-api-54d89bd5fc-cbrc5\" (UID: \"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:58.417251 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:58.417215 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e21ab9-d7e9-4f0d-9a03-8699c11061b6-tls-certs\") pod \"model-serving-api-54d89bd5fc-cbrc5\" (UID: \"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6\") " pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:58.608322 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:58.608197 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:52:58.791386 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:58.791354 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cbrc5"] Jul 06 22:52:58.794071 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:52:58.794027 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e21ab9_d7e9_4f0d_9a03_8699c11061b6.slice/crio-18569877fba4243741662df20711b5c2bb003a0eca39a26ef07bb71ea05a1865 WatchSource:0}: Error finding container 18569877fba4243741662df20711b5c2bb003a0eca39a26ef07bb71ea05a1865: Status 404 returned error can't find the container with id 18569877fba4243741662df20711b5c2bb003a0eca39a26ef07bb71ea05a1865 Jul 06 22:52:59.058837 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.058735 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" event={"ID":"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6","Type":"ContainerStarted","Data":"18569877fba4243741662df20711b5c2bb003a0eca39a26ef07bb71ea05a1865"} Jul 06 22:52:59.083845 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.083804 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-dndq2"] Jul 06 22:52:59.090570 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.090539 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.098802 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.098542 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 22:52:59.098802 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.098542 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 22:52:59.098802 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.098740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-66zt8\"" Jul 06 22:52:59.099094 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.098903 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 22:52:59.099486 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.099463 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 22:52:59.140393 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.140351 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-dndq2"] Jul 06 22:52:59.222552 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.222508 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7fed50-930a-44e8-89e4-5e64dbd19346-local-certs\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.222761 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.222557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.222761 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.222624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-token\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.222761 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.222681 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.222761 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.222740 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-cacerts\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.222944 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.222780 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.222944 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.222807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf577\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-kube-api-access-cf577\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.323768 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.323657 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7fed50-930a-44e8-89e4-5e64dbd19346-local-certs\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.323768 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.323739 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.324004 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.323782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-token\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.324004 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.323828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.324004 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.323859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-cacerts\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.324004 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.323902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.324004 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.323929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf577\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-kube-api-access-cf577\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.324495 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.324446 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.326505 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.326472 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7fed50-930a-44e8-89e4-5e64dbd19346-local-certs\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.326850 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.326817 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.326991 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.326961 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.327054 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.327009 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-cacerts\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.341332 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.341266 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-token\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.343344 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.343307 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf577\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-kube-api-access-cf577\") pod \"istiod-openshift-gateway-7574459656-dndq2\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:52:59.403389 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:52:59.403354 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:53:01.149506 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:01.149465 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-dndq2"] Jul 06 22:53:01.153559 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:53:01.153437 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7fed50_930a_44e8_89e4_5e64dbd19346.slice/crio-3bf9fc673e767690c59d5bbf1086381219c0b823fee83702666aa97fa9743307 WatchSource:0}: Error finding container 3bf9fc673e767690c59d5bbf1086381219c0b823fee83702666aa97fa9743307: Status 404 returned error can't find the container with id 3bf9fc673e767690c59d5bbf1086381219c0b823fee83702666aa97fa9743307 Jul 06 22:53:02.078489 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.078439 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" event={"ID":"650d898d-1738-4b98-b1df-7ce751e04ab8","Type":"ContainerStarted","Data":"77677a05e34693437286a52dc72b09d2153202d5f77b2162b38bee4b3486f513"} Jul 06 22:53:02.078704 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.078509 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:53:02.079954 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.079921 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" event={"ID":"4b7fed50-930a-44e8-89e4-5e64dbd19346","Type":"ContainerStarted","Data":"3bf9fc673e767690c59d5bbf1086381219c0b823fee83702666aa97fa9743307"} Jul 06 22:53:02.081530 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.081501 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" event={"ID":"e20fa170-f482-45a3-ac02-391f1a736daf","Type":"ContainerStarted","Data":"e0dcad7f540af8340fb0ea0ca4177feaa665142a0d9209174321f119394b6603"} Jul 06 22:53:02.083246 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.083220 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" event={"ID":"a1e21ab9-d7e9-4f0d-9a03-8699c11061b6","Type":"ContainerStarted","Data":"66734a8902f23d3798f5f2a263160a63a296d5c13d08678d6bcfe72d3f6bf671"} Jul 06 22:53:02.083382 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.083365 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:53:02.098320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.098262 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" podStartSLOduration=2.360650302 podStartE2EDuration="6.098245887s" totalImagesPullingTime="3.737595585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:02.096833617 +0000 UTC m=+395.396535240" watchObservedRunningTime="2026-07-06 22:53:02.098245887 +0000 UTC m=+395.397947489" Jul 06 22:53:02.117587 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.117524 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" podStartSLOduration=2.303645489 podStartE2EDuration="5.117508691s" totalImagesPullingTime="2.813863202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:02.1165298 +0000 UTC m=+395.416231402" watchObservedRunningTime="2026-07-06 22:53:02.117508691 +0000 UTC m=+395.417210293" Jul 06 22:53:02.133963 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:02.133897 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54f95589f8-jqc4s" podStartSLOduration=2.241040982 podStartE2EDuration="6.133874651s" totalImagesPullingTime="3.892833669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:02.13332976 +0000 UTC m=+395.433031383" watchObservedRunningTime="2026-07-06 22:53:02.133874651 +0000 UTC m=+395.433576253" Jul 06 22:53:03.421053 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:03.421006 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:53:03.421374 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:03.421080 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:53:04.095999 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:04.095954 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" event={"ID":"4b7fed50-930a-44e8-89e4-5e64dbd19346","Type":"ContainerStarted","Data":"81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77"} Jul 06 22:53:04.096259 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:04.096236 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:53:04.098196 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:04.098162 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-dndq2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 22:53:04.098324 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:04.098226 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" podUID="4b7fed50-930a-44e8-89e4-5e64dbd19346" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:53:04.133477 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:04.133420 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" podStartSLOduration=2.868391996 podStartE2EDuration="5.133402512s" totalImagesPullingTime="2.265010516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:04.130741955 +0000 UTC m=+397.430443584" watchObservedRunningTime="2026-07-06 22:53:04.133402512 +0000 UTC m=+397.433104114" Jul 06 22:53:05.101012 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.100979 2561 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-dndq2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 22:53:05.101397 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.101046 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" podUID="4b7fed50-930a-44e8-89e4-5e64dbd19346" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:53:05.420060 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.420029 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7"] Jul 06 22:53:05.424083 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.424050 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.427350 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.427320 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dv6gn\"" Jul 06 22:53:05.441334 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.441300 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7"] Jul 06 22:53:05.483537 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483500 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483704 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483572 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483704 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483604 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483704 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483704 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483659 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483704 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483704 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483944 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483719 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwwp\" (UniqueName: \"kubernetes.io/projected/cdeabf53-1e52-4feb-9cb4-dce852a82f64-kube-api-access-sgwwp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.483944 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.483754 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585149 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585089 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585306 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585306 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585238 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585306 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585273 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585483 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585483 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585334 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585483 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585428 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585623 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585480 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwwp\" (UniqueName: \"kubernetes.io/projected/cdeabf53-1e52-4feb-9cb4-dce852a82f64-kube-api-access-sgwwp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585623 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585515 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.585623 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.585587 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.586145 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.586058 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.586145 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.586096 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.586323 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.586187 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.586323 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.586277 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.587566 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.587546 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.588410 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.588390 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.602964 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.602927 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdeabf53-1e52-4feb-9cb4-dce852a82f64-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.603099 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.603013 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwwp\" (UniqueName: \"kubernetes.io/projected/cdeabf53-1e52-4feb-9cb4-dce852a82f64-kube-api-access-sgwwp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678d5bp7\" (UID: \"cdeabf53-1e52-4feb-9cb4-dce852a82f64\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.734799 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.734707 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:05.924579 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:05.924539 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7"] Jul 06 22:53:05.933226 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:53:05.933199 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdeabf53_1e52_4feb_9cb4_dce852a82f64.slice/crio-dc44eb22b686e4cb3879f4a20d1fd5e1a50a5f22962b13748b0d86cc56ecf300 WatchSource:0}: Error finding container dc44eb22b686e4cb3879f4a20d1fd5e1a50a5f22962b13748b0d86cc56ecf300: Status 404 returned error can't find the container with id dc44eb22b686e4cb3879f4a20d1fd5e1a50a5f22962b13748b0d86cc56ecf300 Jul 06 22:53:06.105996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:06.105959 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" event={"ID":"cdeabf53-1e52-4feb-9cb4-dce852a82f64","Type":"ContainerStarted","Data":"dc44eb22b686e4cb3879f4a20d1fd5e1a50a5f22962b13748b0d86cc56ecf300"} Jul 06 22:53:07.011637 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.011599 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57"] Jul 06 22:53:07.016780 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.016745 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.020187 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.020155 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wkz4c\"" Jul 06 22:53:07.020421 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.020401 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 22:53:07.051297 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.051262 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57"] Jul 06 22:53:07.098734 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.098672 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzsvz\" (UniqueName: \"kubernetes.io/projected/2ebebcc1-626f-4f0d-a317-b1c8edbb4349-kube-api-access-fzsvz\") pod \"llmisvc-controller-manager-777c6bd7d5-hpp57\" (UID: \"2ebebcc1-626f-4f0d-a317-b1c8edbb4349\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.098734 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.098737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ebebcc1-626f-4f0d-a317-b1c8edbb4349-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hpp57\" (UID: \"2ebebcc1-626f-4f0d-a317-b1c8edbb4349\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.199626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.199585 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzsvz\" (UniqueName: \"kubernetes.io/projected/2ebebcc1-626f-4f0d-a317-b1c8edbb4349-kube-api-access-fzsvz\") pod \"llmisvc-controller-manager-777c6bd7d5-hpp57\" (UID: \"2ebebcc1-626f-4f0d-a317-b1c8edbb4349\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.199626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.199627 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ebebcc1-626f-4f0d-a317-b1c8edbb4349-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hpp57\" (UID: \"2ebebcc1-626f-4f0d-a317-b1c8edbb4349\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.202102 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.202075 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ebebcc1-626f-4f0d-a317-b1c8edbb4349-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hpp57\" (UID: \"2ebebcc1-626f-4f0d-a317-b1c8edbb4349\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.230189 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.230156 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzsvz\" (UniqueName: \"kubernetes.io/projected/2ebebcc1-626f-4f0d-a317-b1c8edbb4349-kube-api-access-fzsvz\") pod \"llmisvc-controller-manager-777c6bd7d5-hpp57\" (UID: \"2ebebcc1-626f-4f0d-a317-b1c8edbb4349\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.327522 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.327430 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:07.534008 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.533983 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57"] Jul 06 22:53:07.535594 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:53:07.535566 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2ebebcc1_626f_4f0d_a317_b1c8edbb4349.slice/crio-9f3039222929e9ee130cbfb37041f87416b117bff6aa36978c89bd182d24a9da WatchSource:0}: Error finding container 9f3039222929e9ee130cbfb37041f87416b117bff6aa36978c89bd182d24a9da: Status 404 returned error can't find the container with id 9f3039222929e9ee130cbfb37041f87416b117bff6aa36978c89bd182d24a9da Jul 06 22:53:07.822705 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.822654 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-dndq2"] Jul 06 22:53:07.822892 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.822843 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" podUID="4b7fed50-930a-44e8-89e4-5e64dbd19346" containerName="discovery" containerID="cri-o://81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77" gracePeriod=30 Jul 06 22:53:07.823680 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:07.823648 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:53:08.082940 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.082860 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:53:08.108009 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.107974 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-ca-configmap\") pod \"4b7fed50-930a-44e8-89e4-5e64dbd19346\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " Jul 06 22:53:08.108184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108021 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-cacerts\") pod \"4b7fed50-930a-44e8-89e4-5e64dbd19346\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " Jul 06 22:53:08.108184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108066 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7fed50-930a-44e8-89e4-5e64dbd19346-local-certs\") pod \"4b7fed50-930a-44e8-89e4-5e64dbd19346\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " Jul 06 22:53:08.108184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108111 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-dns-cert\") pod \"4b7fed50-930a-44e8-89e4-5e64dbd19346\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " Jul 06 22:53:08.108184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108147 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-token\") pod \"4b7fed50-930a-44e8-89e4-5e64dbd19346\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " Jul 06 22:53:08.108184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108183 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf577\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-kube-api-access-cf577\") pod \"4b7fed50-930a-44e8-89e4-5e64dbd19346\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " Jul 06 22:53:08.108439 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108273 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-kubeconfig\") pod \"4b7fed50-930a-44e8-89e4-5e64dbd19346\" (UID: \"4b7fed50-930a-44e8-89e4-5e64dbd19346\") " Jul 06 22:53:08.108439 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108418 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4b7fed50-930a-44e8-89e4-5e64dbd19346" (UID: "4b7fed50-930a-44e8-89e4-5e64dbd19346"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:53:08.108612 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.108579 2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-ca-configmap\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:53:08.111828 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.111788 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4b7fed50-930a-44e8-89e4-5e64dbd19346" (UID: "4b7fed50-930a-44e8-89e4-5e64dbd19346"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:53:08.111958 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.111929 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-cacerts" (OuterVolumeSpecName: "cacerts") pod "4b7fed50-930a-44e8-89e4-5e64dbd19346" (UID: "4b7fed50-930a-44e8-89e4-5e64dbd19346"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:53:08.112467 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.112431 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7fed50-930a-44e8-89e4-5e64dbd19346-local-certs" (OuterVolumeSpecName: "local-certs") pod "4b7fed50-930a-44e8-89e4-5e64dbd19346" (UID: "4b7fed50-930a-44e8-89e4-5e64dbd19346"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:08.112988 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.112717 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4b7fed50-930a-44e8-89e4-5e64dbd19346" (UID: "4b7fed50-930a-44e8-89e4-5e64dbd19346"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:53:08.114938 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.114675 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b7fed50-930a-44e8-89e4-5e64dbd19346" (UID: "4b7fed50-930a-44e8-89e4-5e64dbd19346"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:08.114938 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.114861 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-kube-api-access-cf577" (OuterVolumeSpecName: "kube-api-access-cf577") pod "4b7fed50-930a-44e8-89e4-5e64dbd19346" (UID: "4b7fed50-930a-44e8-89e4-5e64dbd19346"). InnerVolumeSpecName "kube-api-access-cf577". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:08.131311 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.131277 2561 generic.go:356] "Generic (PLEG): container finished" podID="4b7fed50-930a-44e8-89e4-5e64dbd19346" containerID="81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77" exitCode=0 Jul 06 22:53:08.131477 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.131363 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" event={"ID":"4b7fed50-930a-44e8-89e4-5e64dbd19346","Type":"ContainerDied","Data":"81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77"} Jul 06 22:53:08.131477 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.131386 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" Jul 06 22:53:08.131477 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.131454 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-dndq2" event={"ID":"4b7fed50-930a-44e8-89e4-5e64dbd19346","Type":"ContainerDied","Data":"3bf9fc673e767690c59d5bbf1086381219c0b823fee83702666aa97fa9743307"} Jul 06 22:53:08.131643 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.131485 2561 scope.go:117] "RemoveContainer" containerID="81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77" Jul 06 22:53:08.132954 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.132895 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" event={"ID":"2ebebcc1-626f-4f0d-a317-b1c8edbb4349","Type":"ContainerStarted","Data":"9f3039222929e9ee130cbfb37041f87416b117bff6aa36978c89bd182d24a9da"} Jul 06 22:53:08.162339 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.162306 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-dndq2"] Jul 06 22:53:08.166612 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.166580 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-dndq2"] Jul 06 22:53:08.185157 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.185134 2561 scope.go:117] "RemoveContainer" containerID="81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77" Jul 06 22:53:08.187965 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:53:08.185787 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77\": container with ID starting with 81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77 not found: ID does not exist" containerID="81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77" Jul 06 22:53:08.187965 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.185828 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77"} err="failed to get container status \"81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77\": rpc error: code = NotFound desc = could not find container \"81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77\": container with ID starting with 81d9a003f462b1bb9f40a97461f0e6ad872b501860e57388c9baedc48a392f77 not found: ID does not exist" Jul 06 22:53:08.209583 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.209547 2561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7fed50-930a-44e8-89e4-5e64dbd19346-local-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:53:08.209583 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.209576 2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-csr-dns-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:53:08.209583 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.209591 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:53:08.210103 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.209600 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf577\" (UniqueName: \"kubernetes.io/projected/4b7fed50-930a-44e8-89e4-5e64dbd19346-kube-api-access-cf577\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:53:08.210103 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.209610 2561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-istio-kubeconfig\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:53:08.210103 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:08.209622 2561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4b7fed50-930a-44e8-89e4-5e64dbd19346-cacerts\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:53:09.220788 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:09.220752 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b7fed50-930a-44e8-89e4-5e64dbd19346" path="/var/lib/kubelet/pods/4b7fed50-930a-44e8-89e4-5e64dbd19346/volumes" Jul 06 22:53:11.151794 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:11.151752 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" event={"ID":"2ebebcc1-626f-4f0d-a317-b1c8edbb4349","Type":"ContainerStarted","Data":"e2a93fb8cb6cf51e9c1e569893f7a140088b5190cf69a69d67e6840fce9f24df"} Jul 06 22:53:11.152204 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:11.151854 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:11.187885 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:11.187757 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" podStartSLOduration=2.188279618 podStartE2EDuration="5.187732948s" totalImagesPullingTime="2.99945333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:11.18678792 +0000 UTC m=+404.486489541" watchObservedRunningTime="2026-07-06 22:53:11.187732948 +0000 UTC m=+404.487434551" Jul 06 22:53:13.091040 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:13.091008 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wvqp8" Jul 06 22:53:13.092959 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:13.092935 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cbrc5" Jul 06 22:53:17.495874 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:17.495829 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:53:17.496277 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:17.495924 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:53:17.496277 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:17.495976 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:53:18.189534 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:18.189495 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" event={"ID":"cdeabf53-1e52-4feb-9cb4-dce852a82f64","Type":"ContainerStarted","Data":"61ad7054d8df4855981ef985119abcbd05d5b4246e1e194b7177032ec089ff65"} Jul 06 22:53:18.214552 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:18.214495 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" podStartSLOduration=1.654207521 podStartE2EDuration="13.214478735s" totalImagesPullingTime="11.560271214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:18.213250291 +0000 UTC m=+411.512951917" watchObservedRunningTime="2026-07-06 22:53:18.214478735 +0000 UTC m=+411.514180338" Jul 06 22:53:18.735255 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:18.735212 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:18.740804 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:18.740772 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:19.194042 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:19.194011 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:19.194960 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:19.194936 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678d5bp7" Jul 06 22:53:42.159148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:42.159112 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hpp57" Jul 06 22:53:57.269653 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.269618 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chbh2"] Jul 06 22:53:57.270164 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.269949 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b7fed50-930a-44e8-89e4-5e64dbd19346" containerName="discovery" Jul 06 22:53:57.270164 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.269961 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b7fed50-930a-44e8-89e4-5e64dbd19346" containerName="discovery" Jul 06 22:53:57.270164 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.270020 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b7fed50-930a-44e8-89e4-5e64dbd19346" containerName="discovery" Jul 06 22:53:57.273059 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.273038 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" Jul 06 22:53:57.275842 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.275814 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 22:53:57.275974 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.275844 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qsc49\"" Jul 06 22:53:57.276911 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.276890 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 22:53:57.286012 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.285986 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chbh2"] Jul 06 22:53:57.337008 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.336974 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5qmx\" (UniqueName: \"kubernetes.io/projected/763459a2-5711-43e2-8a62-9630b91baa89-kube-api-access-s5qmx\") pod \"kuadrant-operator-catalog-chbh2\" (UID: \"763459a2-5711-43e2-8a62-9630b91baa89\") " pod="kuadrant-system/kuadrant-operator-catalog-chbh2" Jul 06 22:53:57.437995 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.437954 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5qmx\" (UniqueName: \"kubernetes.io/projected/763459a2-5711-43e2-8a62-9630b91baa89-kube-api-access-s5qmx\") pod \"kuadrant-operator-catalog-chbh2\" (UID: \"763459a2-5711-43e2-8a62-9630b91baa89\") " pod="kuadrant-system/kuadrant-operator-catalog-chbh2" Jul 06 22:53:57.445982 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.445950 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5qmx\" (UniqueName: \"kubernetes.io/projected/763459a2-5711-43e2-8a62-9630b91baa89-kube-api-access-s5qmx\") pod \"kuadrant-operator-catalog-chbh2\" (UID: \"763459a2-5711-43e2-8a62-9630b91baa89\") " pod="kuadrant-system/kuadrant-operator-catalog-chbh2" Jul 06 22:53:57.585292 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.585193 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" Jul 06 22:53:57.621815 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.621777 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chbh2"] Jul 06 22:53:57.733174 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.733131 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chbh2"] Jul 06 22:53:57.735711 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:53:57.735660 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763459a2_5711_43e2_8a62_9630b91baa89.slice/crio-8c47fa5b0ed6d87c2558c7d0903212aef3e21af4abdcf18748162b6438b4b560 WatchSource:0}: Error finding container 8c47fa5b0ed6d87c2558c7d0903212aef3e21af4abdcf18748162b6438b4b560: Status 404 returned error can't find the container with id 8c47fa5b0ed6d87c2558c7d0903212aef3e21af4abdcf18748162b6438b4b560 Jul 06 22:53:57.858459 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.858369 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p74df"] Jul 06 22:53:57.862843 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.862823 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:53:57.879397 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.879366 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p74df"] Jul 06 22:53:57.942838 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:57.942795 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fngq8\" (UniqueName: \"kubernetes.io/projected/c9006499-1789-4d73-99dd-0f422c4c8a3f-kube-api-access-fngq8\") pod \"kuadrant-operator-catalog-p74df\" (UID: \"c9006499-1789-4d73-99dd-0f422c4c8a3f\") " pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:53:58.044081 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:58.044043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fngq8\" (UniqueName: \"kubernetes.io/projected/c9006499-1789-4d73-99dd-0f422c4c8a3f-kube-api-access-fngq8\") pod \"kuadrant-operator-catalog-p74df\" (UID: \"c9006499-1789-4d73-99dd-0f422c4c8a3f\") " pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:53:58.052037 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:58.052007 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fngq8\" (UniqueName: \"kubernetes.io/projected/c9006499-1789-4d73-99dd-0f422c4c8a3f-kube-api-access-fngq8\") pod \"kuadrant-operator-catalog-p74df\" (UID: \"c9006499-1789-4d73-99dd-0f422c4c8a3f\") " pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:53:58.172205 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:58.172176 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:53:58.309058 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:58.309028 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p74df"] Jul 06 22:53:58.311270 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:53:58.311248 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9006499_1789_4d73_99dd_0f422c4c8a3f.slice/crio-6b10ddb2fa9a3de8fa593ea56228afbc4f2dc1b28a9a4173c92b98dcb2f981ba WatchSource:0}: Error finding container 6b10ddb2fa9a3de8fa593ea56228afbc4f2dc1b28a9a4173c92b98dcb2f981ba: Status 404 returned error can't find the container with id 6b10ddb2fa9a3de8fa593ea56228afbc4f2dc1b28a9a4173c92b98dcb2f981ba Jul 06 22:53:58.372907 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:58.372858 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p74df" event={"ID":"c9006499-1789-4d73-99dd-0f422c4c8a3f","Type":"ContainerStarted","Data":"6b10ddb2fa9a3de8fa593ea56228afbc4f2dc1b28a9a4173c92b98dcb2f981ba"} Jul 06 22:53:58.374087 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:53:58.374050 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" event={"ID":"763459a2-5711-43e2-8a62-9630b91baa89","Type":"ContainerStarted","Data":"8c47fa5b0ed6d87c2558c7d0903212aef3e21af4abdcf18748162b6438b4b560"} Jul 06 22:54:00.390810 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.390311 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" event={"ID":"763459a2-5711-43e2-8a62-9630b91baa89","Type":"ContainerStarted","Data":"a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74"} Jul 06 22:54:00.390810 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.390362 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" podUID="763459a2-5711-43e2-8a62-9630b91baa89" containerName="registry-server" containerID="cri-o://a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74" gracePeriod=2 Jul 06 22:54:00.392082 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.392049 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p74df" event={"ID":"c9006499-1789-4d73-99dd-0f422c4c8a3f","Type":"ContainerStarted","Data":"61822942ee01977089884dcb9cc70872bf49c2676edc5990d49d184d5bfa9996"} Jul 06 22:54:00.409178 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.409124 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" podStartSLOduration=0.868972857 podStartE2EDuration="3.409104754s" totalImagesPullingTime="2.540131897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:00.406570136 +0000 UTC m=+453.706271739" watchObservedRunningTime="2026-07-06 22:54:00.409104754 +0000 UTC m=+453.708806355" Jul 06 22:54:00.423108 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.423054 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-p74df" podStartSLOduration=1.454361896 podStartE2EDuration="3.423037213s" totalImagesPullingTime="1.968675317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:00.422113512 +0000 UTC m=+453.721815116" watchObservedRunningTime="2026-07-06 22:54:00.423037213 +0000 UTC m=+453.722738814" Jul 06 22:54:00.638098 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.638072 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-chbh2_763459a2-5711-43e2-8a62-9630b91baa89/registry-server/0.log" Jul 06 22:54:00.640428 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.640409 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" Jul 06 22:54:00.669013 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.668980 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5qmx\" (UniqueName: \"kubernetes.io/projected/763459a2-5711-43e2-8a62-9630b91baa89-kube-api-access-s5qmx\") pod \"763459a2-5711-43e2-8a62-9630b91baa89\" (UID: \"763459a2-5711-43e2-8a62-9630b91baa89\") " Jul 06 22:54:00.671246 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.671220 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/763459a2-5711-43e2-8a62-9630b91baa89-kube-api-access-s5qmx" (OuterVolumeSpecName: "kube-api-access-s5qmx") pod "763459a2-5711-43e2-8a62-9630b91baa89" (UID: "763459a2-5711-43e2-8a62-9630b91baa89"). InnerVolumeSpecName "kube-api-access-s5qmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:54:00.770250 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:00.770210 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5qmx\" (UniqueName: \"kubernetes.io/projected/763459a2-5711-43e2-8a62-9630b91baa89-kube-api-access-s5qmx\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:01.398160 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.398129 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-chbh2_763459a2-5711-43e2-8a62-9630b91baa89/registry-server/0.log" Jul 06 22:54:01.398585 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.398174 2561 generic.go:356] "Generic (PLEG): container finished" podID="763459a2-5711-43e2-8a62-9630b91baa89" containerID="a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74" exitCode=1 Jul 06 22:54:01.398585 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.398230 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" Jul 06 22:54:01.398585 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.398263 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" event={"ID":"763459a2-5711-43e2-8a62-9630b91baa89","Type":"ContainerDied","Data":"a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74"} Jul 06 22:54:01.398585 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.398316 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chbh2" event={"ID":"763459a2-5711-43e2-8a62-9630b91baa89","Type":"ContainerDied","Data":"8c47fa5b0ed6d87c2558c7d0903212aef3e21af4abdcf18748162b6438b4b560"} Jul 06 22:54:01.398585 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.398336 2561 scope.go:117] "RemoveContainer" containerID="a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74" Jul 06 22:54:01.407343 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.407321 2561 scope.go:117] "RemoveContainer" containerID="a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74" Jul 06 22:54:01.407659 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:01.407635 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74\": container with ID starting with a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74 not found: ID does not exist" containerID="a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74" Jul 06 22:54:01.407739 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.407671 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74"} err="failed to get container status \"a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74\": rpc error: code = NotFound desc = could not find container \"a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74\": container with ID starting with a7accd9a28d9424667ca39a686c958b42b249029a21e6b25420f2a6957b9db74 not found: ID does not exist" Jul 06 22:54:01.417894 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.417860 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chbh2"] Jul 06 22:54:01.422242 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:01.422209 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chbh2"] Jul 06 22:54:03.220483 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:03.220441 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="763459a2-5711-43e2-8a62-9630b91baa89" path="/var/lib/kubelet/pods/763459a2-5711-43e2-8a62-9630b91baa89/volumes" Jul 06 22:54:08.172804 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:08.172758 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:54:08.173183 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:08.172837 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:54:08.196180 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:08.196151 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:54:08.453132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:08.453054 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-p74df" Jul 06 22:54:12.791108 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.791067 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2"] Jul 06 22:54:12.791495 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.791384 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="763459a2-5711-43e2-8a62-9630b91baa89" containerName="registry-server" Jul 06 22:54:12.791495 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.791394 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="763459a2-5711-43e2-8a62-9630b91baa89" containerName="registry-server" Jul 06 22:54:12.791495 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.791465 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="763459a2-5711-43e2-8a62-9630b91baa89" containerName="registry-server" Jul 06 22:54:12.796142 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.796117 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.798871 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.798839 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jmlxb\"" Jul 06 22:54:12.803868 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.803838 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2"] Jul 06 22:54:12.873749 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.873677 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvjmb\" (UniqueName: \"kubernetes.io/projected/7e30516f-e531-4072-a38b-67fe2f38ff4e-kube-api-access-rvjmb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.873931 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.873826 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.873931 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.873898 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.974529 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.974490 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.974745 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.974567 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.974745 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.974619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvjmb\" (UniqueName: \"kubernetes.io/projected/7e30516f-e531-4072-a38b-67fe2f38ff4e-kube-api-access-rvjmb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.974958 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.974937 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.975036 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.974958 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:12.982446 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:12.982420 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvjmb\" (UniqueName: \"kubernetes.io/projected/7e30516f-e531-4072-a38b-67fe2f38ff4e-kube-api-access-rvjmb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:13.106592 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.106494 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:13.249093 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.249068 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2"] Jul 06 22:54:13.251311 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:13.251286 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e30516f_e531_4072_a38b_67fe2f38ff4e.slice/crio-7d6fbe3d3528fcdcb647f4ea2316de968a84a4024df31e321ac65fc0c7be2f3e WatchSource:0}: Error finding container 7d6fbe3d3528fcdcb647f4ea2316de968a84a4024df31e321ac65fc0c7be2f3e: Status 404 returned error can't find the container with id 7d6fbe3d3528fcdcb647f4ea2316de968a84a4024df31e321ac65fc0c7be2f3e Jul 06 22:54:13.394012 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.393926 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb"] Jul 06 22:54:13.397351 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.397330 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.405414 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.405384 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb"] Jul 06 22:54:13.456314 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.456274 2561 generic.go:356] "Generic (PLEG): container finished" podID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerID="8d05c9ea43027ad396a32cab95c74840768be5c660cf0e3b8ba0607516031999" exitCode=0 Jul 06 22:54:13.456490 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.456318 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" event={"ID":"7e30516f-e531-4072-a38b-67fe2f38ff4e","Type":"ContainerDied","Data":"8d05c9ea43027ad396a32cab95c74840768be5c660cf0e3b8ba0607516031999"} Jul 06 22:54:13.456490 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.456378 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" event={"ID":"7e30516f-e531-4072-a38b-67fe2f38ff4e","Type":"ContainerStarted","Data":"7d6fbe3d3528fcdcb647f4ea2316de968a84a4024df31e321ac65fc0c7be2f3e"} Jul 06 22:54:13.479140 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.479100 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.479140 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.479143 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkqbt\" (UniqueName: \"kubernetes.io/projected/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-kube-api-access-xkqbt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.479361 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.479171 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.580091 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.580046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.580287 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.580262 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.580338 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.580307 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkqbt\" (UniqueName: \"kubernetes.io/projected/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-kube-api-access-xkqbt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.580461 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.580439 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.580552 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.580538 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.588237 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.588209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkqbt\" (UniqueName: \"kubernetes.io/projected/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-kube-api-access-xkqbt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.722089 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.722052 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:13.792454 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.792414 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx"] Jul 06 22:54:13.797381 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.797351 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.803628 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.803595 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx"] Jul 06 22:54:13.865032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.864996 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb"] Jul 06 22:54:13.867947 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:13.867912 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d62bc7e_b2fc_4dd4_8b8f_9e4964c83187.slice/crio-56e292c884cfddbbd568f719770c579afc3e7b4fc709741b4c7e910f6a3f744d WatchSource:0}: Error finding container 56e292c884cfddbbd568f719770c579afc3e7b4fc709741b4c7e910f6a3f744d: Status 404 returned error can't find the container with id 56e292c884cfddbbd568f719770c579afc3e7b4fc709741b4c7e910f6a3f744d Jul 06 22:54:13.882532 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.882502 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpjrh\" (UniqueName: \"kubernetes.io/projected/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-kube-api-access-gpjrh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.882641 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.882627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.882723 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.882655 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.983388 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.983300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.983388 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.983356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.983597 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.983424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpjrh\" (UniqueName: \"kubernetes.io/projected/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-kube-api-access-gpjrh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.983838 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.983805 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.983838 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.983828 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:13.991815 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:13.991782 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpjrh\" (UniqueName: \"kubernetes.io/projected/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-kube-api-access-gpjrh\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:14.109880 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.109838 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:14.249216 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.249191 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx"] Jul 06 22:54:14.251249 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:14.251224 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07fcd1e_5d05_4cec_a74c_5cf8464a0499.slice/crio-69965dcf0cf1a6b894028df6360441c471ac713138aeb116b40de5bc8ab9337f WatchSource:0}: Error finding container 69965dcf0cf1a6b894028df6360441c471ac713138aeb116b40de5bc8ab9337f: Status 404 returned error can't find the container with id 69965dcf0cf1a6b894028df6360441c471ac713138aeb116b40de5bc8ab9337f Jul 06 22:54:14.391736 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.391652 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj"] Jul 06 22:54:14.395114 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.395092 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.402707 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.402585 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj"] Jul 06 22:54:14.463808 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.463776 2561 generic.go:356] "Generic (PLEG): container finished" podID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerID="360b66f92f6e317f19685a3b87b06a19fc0924ebed9d3d51c063cb0ccd320fb3" exitCode=0 Jul 06 22:54:14.463979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.463859 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" event={"ID":"d07fcd1e-5d05-4cec-a74c-5cf8464a0499","Type":"ContainerDied","Data":"360b66f92f6e317f19685a3b87b06a19fc0924ebed9d3d51c063cb0ccd320fb3"} Jul 06 22:54:14.463979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.463912 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" event={"ID":"d07fcd1e-5d05-4cec-a74c-5cf8464a0499","Type":"ContainerStarted","Data":"69965dcf0cf1a6b894028df6360441c471ac713138aeb116b40de5bc8ab9337f"} Jul 06 22:54:14.465697 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.465652 2561 generic.go:356] "Generic (PLEG): container finished" podID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerID="3318a508f97ca77c810852db38dc5b01bb6e97fb3a84db7f570a7bc94151dadb" exitCode=0 Jul 06 22:54:14.465820 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.465797 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" event={"ID":"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187","Type":"ContainerDied","Data":"3318a508f97ca77c810852db38dc5b01bb6e97fb3a84db7f570a7bc94151dadb"} Jul 06 22:54:14.465869 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.465853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" event={"ID":"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187","Type":"ContainerStarted","Data":"56e292c884cfddbbd568f719770c579afc3e7b4fc709741b4c7e910f6a3f744d"} Jul 06 22:54:14.487566 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.487530 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.487752 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.487602 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.487752 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.487652 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvcf4\" (UniqueName: \"kubernetes.io/projected/6527ff3e-840b-47d3-9500-86dc480b51c3-kube-api-access-xvcf4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.589041 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.588994 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.589242 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.589087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.589242 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.589131 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvcf4\" (UniqueName: \"kubernetes.io/projected/6527ff3e-840b-47d3-9500-86dc480b51c3-kube-api-access-xvcf4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.589383 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.589359 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.589468 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.589445 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.597601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.597567 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvcf4\" (UniqueName: \"kubernetes.io/projected/6527ff3e-840b-47d3-9500-86dc480b51c3-kube-api-access-xvcf4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.746740 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.746697 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:14.885948 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:14.885918 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj"] Jul 06 22:54:14.887730 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:14.887681 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6527ff3e_840b_47d3_9500_86dc480b51c3.slice/crio-17db7921a8e00644a438f1f0c8802e2671eecb07d18ac6b6a28ef5ed23fc2fc0 WatchSource:0}: Error finding container 17db7921a8e00644a438f1f0c8802e2671eecb07d18ac6b6a28ef5ed23fc2fc0: Status 404 returned error can't find the container with id 17db7921a8e00644a438f1f0c8802e2671eecb07d18ac6b6a28ef5ed23fc2fc0 Jul 06 22:54:15.474490 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:15.474457 2561 generic.go:356] "Generic (PLEG): container finished" podID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerID="bc26f5eede6f16c6ac006ae3dd81d6a2151830acbd59253c12e5346ed9a4d0d0" exitCode=0 Jul 06 22:54:15.474664 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:15.474535 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" event={"ID":"6527ff3e-840b-47d3-9500-86dc480b51c3","Type":"ContainerDied","Data":"bc26f5eede6f16c6ac006ae3dd81d6a2151830acbd59253c12e5346ed9a4d0d0"} Jul 06 22:54:15.474664 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:15.474591 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" event={"ID":"6527ff3e-840b-47d3-9500-86dc480b51c3","Type":"ContainerStarted","Data":"17db7921a8e00644a438f1f0c8802e2671eecb07d18ac6b6a28ef5ed23fc2fc0"} Jul 06 22:54:15.476435 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:15.476407 2561 generic.go:356] "Generic (PLEG): container finished" podID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerID="5fb2c04cea36bdd58b41824070ec1f145cc09c850e8a92e95e8ae73b53e8501f" exitCode=0 Jul 06 22:54:15.476545 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:15.476485 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" event={"ID":"7e30516f-e531-4072-a38b-67fe2f38ff4e","Type":"ContainerDied","Data":"5fb2c04cea36bdd58b41824070ec1f145cc09c850e8a92e95e8ae73b53e8501f"} Jul 06 22:54:15.478538 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:15.478515 2561 generic.go:356] "Generic (PLEG): container finished" podID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerID="1feb271428d679f2d120ad19d06a179c6b6f0b085fe9275f7e9d2ed26e393da3" exitCode=0 Jul 06 22:54:15.478642 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:15.478554 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" event={"ID":"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187","Type":"ContainerDied","Data":"1feb271428d679f2d120ad19d06a179c6b6f0b085fe9275f7e9d2ed26e393da3"} Jul 06 22:54:16.485221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.485182 2561 generic.go:356] "Generic (PLEG): container finished" podID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerID="40909f13be7a84e215ff58bf19f63f9ece1d5c1f8e383d7c1432487e897c4c54" exitCode=0 Jul 06 22:54:16.485655 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.485265 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" event={"ID":"6527ff3e-840b-47d3-9500-86dc480b51c3","Type":"ContainerDied","Data":"40909f13be7a84e215ff58bf19f63f9ece1d5c1f8e383d7c1432487e897c4c54"} Jul 06 22:54:16.487542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.487512 2561 generic.go:356] "Generic (PLEG): container finished" podID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerID="130f116dc9435b0f3467482c2802412948e71e1506063d351bacb723a6d5ac2c" exitCode=0 Jul 06 22:54:16.487647 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.487591 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" event={"ID":"7e30516f-e531-4072-a38b-67fe2f38ff4e","Type":"ContainerDied","Data":"130f116dc9435b0f3467482c2802412948e71e1506063d351bacb723a6d5ac2c"} Jul 06 22:54:16.489509 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.489485 2561 generic.go:356] "Generic (PLEG): container finished" podID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerID="cac7476bca3b169e60e4bc606fa5a157cebb3e542b6caaf5a8fdf0e32e806075" exitCode=0 Jul 06 22:54:16.489591 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.489560 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" event={"ID":"d07fcd1e-5d05-4cec-a74c-5cf8464a0499","Type":"ContainerDied","Data":"cac7476bca3b169e60e4bc606fa5a157cebb3e542b6caaf5a8fdf0e32e806075"} Jul 06 22:54:16.491880 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.491857 2561 generic.go:356] "Generic (PLEG): container finished" podID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerID="ae70c261d55382a246184e8991725ec0478954742e5fb48e5a7062140ba51833" exitCode=0 Jul 06 22:54:16.491966 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:16.491902 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" event={"ID":"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187","Type":"ContainerDied","Data":"ae70c261d55382a246184e8991725ec0478954742e5fb48e5a7062140ba51833"} Jul 06 22:54:17.500046 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.500007 2561 generic.go:356] "Generic (PLEG): container finished" podID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerID="22e91bd78143fd6bdb4a64e88af7c41268f4404b5df3053cb562137d476ca424" exitCode=0 Jul 06 22:54:17.500448 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.500098 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" event={"ID":"6527ff3e-840b-47d3-9500-86dc480b51c3","Type":"ContainerDied","Data":"22e91bd78143fd6bdb4a64e88af7c41268f4404b5df3053cb562137d476ca424"} Jul 06 22:54:17.502152 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.502124 2561 generic.go:356] "Generic (PLEG): container finished" podID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerID="1389b3edc2a71e8901f26567138cd9178c7157ff727ee80c05699e9da218322c" exitCode=0 Jul 06 22:54:17.502278 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.502205 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" event={"ID":"d07fcd1e-5d05-4cec-a74c-5cf8464a0499","Type":"ContainerDied","Data":"1389b3edc2a71e8901f26567138cd9178c7157ff727ee80c05699e9da218322c"} Jul 06 22:54:17.639242 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.639213 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:17.676938 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.676913 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:17.720750 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.720709 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-bundle\") pod \"7e30516f-e531-4072-a38b-67fe2f38ff4e\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " Jul 06 22:54:17.720899 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.720748 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-util\") pod \"7e30516f-e531-4072-a38b-67fe2f38ff4e\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " Jul 06 22:54:17.720899 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.720853 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-bundle\") pod \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " Jul 06 22:54:17.720899 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.720884 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkqbt\" (UniqueName: \"kubernetes.io/projected/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-kube-api-access-xkqbt\") pod \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " Jul 06 22:54:17.721027 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.720937 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvjmb\" (UniqueName: \"kubernetes.io/projected/7e30516f-e531-4072-a38b-67fe2f38ff4e-kube-api-access-rvjmb\") pod \"7e30516f-e531-4072-a38b-67fe2f38ff4e\" (UID: \"7e30516f-e531-4072-a38b-67fe2f38ff4e\") " Jul 06 22:54:17.721027 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.720957 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-util\") pod \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\" (UID: \"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187\") " Jul 06 22:54:17.721508 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.721478 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-bundle" (OuterVolumeSpecName: "bundle") pod "7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" (UID: "7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:17.721647 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.721623 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-bundle" (OuterVolumeSpecName: "bundle") pod "7e30516f-e531-4072-a38b-67fe2f38ff4e" (UID: "7e30516f-e531-4072-a38b-67fe2f38ff4e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:17.723122 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.723089 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-kube-api-access-xkqbt" (OuterVolumeSpecName: "kube-api-access-xkqbt") pod "7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" (UID: "7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187"). InnerVolumeSpecName "kube-api-access-xkqbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:54:17.723231 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.723135 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e30516f-e531-4072-a38b-67fe2f38ff4e-kube-api-access-rvjmb" (OuterVolumeSpecName: "kube-api-access-rvjmb") pod "7e30516f-e531-4072-a38b-67fe2f38ff4e" (UID: "7e30516f-e531-4072-a38b-67fe2f38ff4e"). InnerVolumeSpecName "kube-api-access-rvjmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:54:17.726820 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.726787 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-util" (OuterVolumeSpecName: "util") pod "7e30516f-e531-4072-a38b-67fe2f38ff4e" (UID: "7e30516f-e531-4072-a38b-67fe2f38ff4e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:17.727300 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.727274 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-util" (OuterVolumeSpecName: "util") pod "7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" (UID: "7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:17.822115 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.822013 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:17.822115 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.822051 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e30516f-e531-4072-a38b-67fe2f38ff4e-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:17.822115 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.822063 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:17.822115 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.822072 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkqbt\" (UniqueName: \"kubernetes.io/projected/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-kube-api-access-xkqbt\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:17.822115 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.822083 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvjmb\" (UniqueName: \"kubernetes.io/projected/7e30516f-e531-4072-a38b-67fe2f38ff4e-kube-api-access-rvjmb\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:17.822115 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:17.822092 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:18.510209 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.510167 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" Jul 06 22:54:18.510669 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.510178 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2" event={"ID":"7e30516f-e531-4072-a38b-67fe2f38ff4e","Type":"ContainerDied","Data":"7d6fbe3d3528fcdcb647f4ea2316de968a84a4024df31e321ac65fc0c7be2f3e"} Jul 06 22:54:18.510669 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.510293 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d6fbe3d3528fcdcb647f4ea2316de968a84a4024df31e321ac65fc0c7be2f3e" Jul 06 22:54:18.512105 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.512085 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" Jul 06 22:54:18.512244 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.512123 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb" event={"ID":"7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187","Type":"ContainerDied","Data":"56e292c884cfddbbd568f719770c579afc3e7b4fc709741b4c7e910f6a3f744d"} Jul 06 22:54:18.512244 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.512159 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56e292c884cfddbbd568f719770c579afc3e7b4fc709741b4c7e910f6a3f744d" Jul 06 22:54:18.686921 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.686885 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:18.688947 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.688916 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:18.833398 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.833286 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpjrh\" (UniqueName: \"kubernetes.io/projected/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-kube-api-access-gpjrh\") pod \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " Jul 06 22:54:18.833398 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.833341 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-util\") pod \"6527ff3e-840b-47d3-9500-86dc480b51c3\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " Jul 06 22:54:18.833398 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.833388 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvcf4\" (UniqueName: \"kubernetes.io/projected/6527ff3e-840b-47d3-9500-86dc480b51c3-kube-api-access-xvcf4\") pod \"6527ff3e-840b-47d3-9500-86dc480b51c3\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " Jul 06 22:54:18.833731 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.833429 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-util\") pod \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " Jul 06 22:54:18.833731 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.833457 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-bundle\") pod \"6527ff3e-840b-47d3-9500-86dc480b51c3\" (UID: \"6527ff3e-840b-47d3-9500-86dc480b51c3\") " Jul 06 22:54:18.833731 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.833494 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-bundle\") pod \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\" (UID: \"d07fcd1e-5d05-4cec-a74c-5cf8464a0499\") " Jul 06 22:54:18.834060 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.834028 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-bundle" (OuterVolumeSpecName: "bundle") pod "d07fcd1e-5d05-4cec-a74c-5cf8464a0499" (UID: "d07fcd1e-5d05-4cec-a74c-5cf8464a0499"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:18.834162 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.834035 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-bundle" (OuterVolumeSpecName: "bundle") pod "6527ff3e-840b-47d3-9500-86dc480b51c3" (UID: "6527ff3e-840b-47d3-9500-86dc480b51c3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:18.835669 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.835633 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-kube-api-access-gpjrh" (OuterVolumeSpecName: "kube-api-access-gpjrh") pod "d07fcd1e-5d05-4cec-a74c-5cf8464a0499" (UID: "d07fcd1e-5d05-4cec-a74c-5cf8464a0499"). InnerVolumeSpecName "kube-api-access-gpjrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:54:18.836009 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.835986 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6527ff3e-840b-47d3-9500-86dc480b51c3-kube-api-access-xvcf4" (OuterVolumeSpecName: "kube-api-access-xvcf4") pod "6527ff3e-840b-47d3-9500-86dc480b51c3" (UID: "6527ff3e-840b-47d3-9500-86dc480b51c3"). InnerVolumeSpecName "kube-api-access-xvcf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:54:18.839836 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.839795 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-util" (OuterVolumeSpecName: "util") pod "d07fcd1e-5d05-4cec-a74c-5cf8464a0499" (UID: "d07fcd1e-5d05-4cec-a74c-5cf8464a0499"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:18.840095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.840071 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-util" (OuterVolumeSpecName: "util") pod "6527ff3e-840b-47d3-9500-86dc480b51c3" (UID: "6527ff3e-840b-47d3-9500-86dc480b51c3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:54:18.934569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.934516 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvcf4\" (UniqueName: \"kubernetes.io/projected/6527ff3e-840b-47d3-9500-86dc480b51c3-kube-api-access-xvcf4\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:18.934569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.934562 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:18.934569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.934572 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:18.934569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.934580 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:18.934867 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.934589 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpjrh\" (UniqueName: \"kubernetes.io/projected/d07fcd1e-5d05-4cec-a74c-5cf8464a0499-kube-api-access-gpjrh\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:18.934867 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:18.934598 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6527ff3e-840b-47d3-9500-86dc480b51c3-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:54:19.519287 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:19.519201 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" Jul 06 22:54:19.519287 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:19.519199 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj" event={"ID":"6527ff3e-840b-47d3-9500-86dc480b51c3","Type":"ContainerDied","Data":"17db7921a8e00644a438f1f0c8802e2671eecb07d18ac6b6a28ef5ed23fc2fc0"} Jul 06 22:54:19.519795 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:19.519321 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17db7921a8e00644a438f1f0c8802e2671eecb07d18ac6b6a28ef5ed23fc2fc0" Jul 06 22:54:19.521244 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:19.521219 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" Jul 06 22:54:19.521380 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:19.521243 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx" event={"ID":"d07fcd1e-5d05-4cec-a74c-5cf8464a0499","Type":"ContainerDied","Data":"69965dcf0cf1a6b894028df6360441c471ac713138aeb116b40de5bc8ab9337f"} Jul 06 22:54:19.521380 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:19.521287 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69965dcf0cf1a6b894028df6360441c471ac713138aeb116b40de5bc8ab9337f" Jul 06 22:54:28.288463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.288417 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7698c68448-dttmw"] Jul 06 22:54:28.288999 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.288973 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerName="util" Jul 06 22:54:28.288999 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.288996 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerName="util" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289010 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerName="util" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289019 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerName="util" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289032 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerName="pull" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289042 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerName="pull" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289052 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerName="pull" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289061 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerName="pull" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289076 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerName="util" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289085 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerName="util" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289098 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerName="pull" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289106 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerName="pull" Jul 06 22:54:28.289119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289118 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerName="util" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289127 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerName="util" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289135 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerName="pull" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289144 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerName="pull" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289154 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289163 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289176 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289184 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289193 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289202 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289214 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289222 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289321 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e30516f-e531-4072-a38b-67fe2f38ff4e" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289332 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289344 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d07fcd1e-5d05-4cec-a74c-5cf8464a0499" containerName="extract" Jul 06 22:54:28.289601 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.289355 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6527ff3e-840b-47d3-9500-86dc480b51c3" containerName="extract" Jul 06 22:54:28.293751 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.293729 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.305703 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.305657 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7698c68448-dttmw"] Jul 06 22:54:28.418949 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.418914 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-service-ca\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.419164 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.418965 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-console-config\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.419164 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.419004 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-trusted-ca-bundle\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.419164 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.419104 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfssw\" (UniqueName: \"kubernetes.io/projected/0fadf873-a021-48a6-9f5f-ee106affe381-kube-api-access-sfssw\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.419164 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.419147 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fadf873-a021-48a6-9f5f-ee106affe381-console-serving-cert\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.419329 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.419174 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fadf873-a021-48a6-9f5f-ee106affe381-console-oauth-config\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.419329 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.419206 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-oauth-serving-cert\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.520275 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.520227 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-service-ca\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.520466 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.520291 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-console-config\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.520466 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.520322 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-trusted-ca-bundle\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.520466 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.520378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfssw\" (UniqueName: \"kubernetes.io/projected/0fadf873-a021-48a6-9f5f-ee106affe381-kube-api-access-sfssw\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.520466 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.520408 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fadf873-a021-48a6-9f5f-ee106affe381-console-serving-cert\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.520657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.520549 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fadf873-a021-48a6-9f5f-ee106affe381-console-oauth-config\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.520657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.520615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-oauth-serving-cert\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.521122 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.521092 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-console-config\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.521241 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.521118 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-service-ca\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.521333 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.521313 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-oauth-serving-cert\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.521399 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.521380 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0fadf873-a021-48a6-9f5f-ee106affe381-trusted-ca-bundle\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.523321 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.523296 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0fadf873-a021-48a6-9f5f-ee106affe381-console-oauth-config\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.523452 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.523432 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0fadf873-a021-48a6-9f5f-ee106affe381-console-serving-cert\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.534780 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.534746 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfssw\" (UniqueName: \"kubernetes.io/projected/0fadf873-a021-48a6-9f5f-ee106affe381-kube-api-access-sfssw\") pod \"console-7698c68448-dttmw\" (UID: \"0fadf873-a021-48a6-9f5f-ee106affe381\") " pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.605567 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.605459 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:28.748748 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:28.748718 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7698c68448-dttmw"] Jul 06 22:54:28.750841 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:28.750813 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fadf873_a021_48a6_9f5f_ee106affe381.slice/crio-74ecfa1bf83ebe3a590748e17e529e067a723ef67a23a7b7f220e9032e447cb7 WatchSource:0}: Error finding container 74ecfa1bf83ebe3a590748e17e529e067a723ef67a23a7b7f220e9032e447cb7: Status 404 returned error can't find the container with id 74ecfa1bf83ebe3a590748e17e529e067a723ef67a23a7b7f220e9032e447cb7 Jul 06 22:54:29.572365 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:29.572324 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7698c68448-dttmw" event={"ID":"0fadf873-a021-48a6-9f5f-ee106affe381","Type":"ContainerStarted","Data":"d5dacaaa3f312859c96638be0ca73a0148fe7c1ed992808dc1947dc1b7b1bbee"} Jul 06 22:54:29.572795 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:29.572379 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7698c68448-dttmw" event={"ID":"0fadf873-a021-48a6-9f5f-ee106affe381","Type":"ContainerStarted","Data":"74ecfa1bf83ebe3a590748e17e529e067a723ef67a23a7b7f220e9032e447cb7"} Jul 06 22:54:29.589283 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:29.589226 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7698c68448-dttmw" podStartSLOduration=1.589209131 podStartE2EDuration="1.589209131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:29.588507702 +0000 UTC m=+482.888209304" watchObservedRunningTime="2026-07-06 22:54:29.589209131 +0000 UTC m=+482.888910732" Jul 06 22:54:31.051849 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.051805 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9"] Jul 06 22:54:31.055210 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.055189 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.059618 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.059595 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7md6w\"" Jul 06 22:54:31.083594 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.083556 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9"] Jul 06 22:54:31.144014 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.143974 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gbz5\" (UniqueName: \"kubernetes.io/projected/1c34442a-ed33-41e0-bbff-2bbaade35279-kube-api-access-5gbz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.144177 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.144090 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c34442a-ed33-41e0-bbff-2bbaade35279-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.244754 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.244715 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gbz5\" (UniqueName: \"kubernetes.io/projected/1c34442a-ed33-41e0-bbff-2bbaade35279-kube-api-access-5gbz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.244933 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.244802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c34442a-ed33-41e0-bbff-2bbaade35279-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.245156 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.245137 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c34442a-ed33-41e0-bbff-2bbaade35279-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.255411 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.255385 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gbz5\" (UniqueName: \"kubernetes.io/projected/1c34442a-ed33-41e0-bbff-2bbaade35279-kube-api-access-5gbz5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.365729 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.365606 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:31.521736 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.521697 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9"] Jul 06 22:54:31.523134 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:31.523102 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c34442a_ed33_41e0_bbff_2bbaade35279.slice/crio-23bb1f4bb5e5b977d0bc0b1da2f70661deb7ab4899fe499f4ea60953cd1962f9 WatchSource:0}: Error finding container 23bb1f4bb5e5b977d0bc0b1da2f70661deb7ab4899fe499f4ea60953cd1962f9: Status 404 returned error can't find the container with id 23bb1f4bb5e5b977d0bc0b1da2f70661deb7ab4899fe499f4ea60953cd1962f9 Jul 06 22:54:31.583584 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:31.583548 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" event={"ID":"1c34442a-ed33-41e0-bbff-2bbaade35279","Type":"ContainerStarted","Data":"23bb1f4bb5e5b977d0bc0b1da2f70661deb7ab4899fe499f4ea60953cd1962f9"} Jul 06 22:54:33.830474 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:33.830433 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt"] Jul 06 22:54:33.835569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:33.835540 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" Jul 06 22:54:33.838332 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:33.838305 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s9hfh\"" Jul 06 22:54:33.851086 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:33.851050 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt"] Jul 06 22:54:33.968426 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:33.968382 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddmbz\" (UniqueName: \"kubernetes.io/projected/3bfc6fc4-7482-46af-bf1d-cead4d5104b9-kube-api-access-ddmbz\") pod \"authorino-operator-75cc8d5cb7-qhhjt\" (UID: \"3bfc6fc4-7482-46af-bf1d-cead4d5104b9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" Jul 06 22:54:34.069764 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:34.069721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmbz\" (UniqueName: \"kubernetes.io/projected/3bfc6fc4-7482-46af-bf1d-cead4d5104b9-kube-api-access-ddmbz\") pod \"authorino-operator-75cc8d5cb7-qhhjt\" (UID: \"3bfc6fc4-7482-46af-bf1d-cead4d5104b9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" Jul 06 22:54:34.090057 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:34.089979 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddmbz\" (UniqueName: \"kubernetes.io/projected/3bfc6fc4-7482-46af-bf1d-cead4d5104b9-kube-api-access-ddmbz\") pod \"authorino-operator-75cc8d5cb7-qhhjt\" (UID: \"3bfc6fc4-7482-46af-bf1d-cead4d5104b9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" Jul 06 22:54:34.148850 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:34.148814 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" Jul 06 22:54:34.320702 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:34.320648 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt"] Jul 06 22:54:34.324055 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:34.324021 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bfc6fc4_7482_46af_bf1d_cead4d5104b9.slice/crio-0551c6cddb50325e0d9ecc18056e09642b44546902fb3a0a8703045aabeabcf7 WatchSource:0}: Error finding container 0551c6cddb50325e0d9ecc18056e09642b44546902fb3a0a8703045aabeabcf7: Status 404 returned error can't find the container with id 0551c6cddb50325e0d9ecc18056e09642b44546902fb3a0a8703045aabeabcf7 Jul 06 22:54:34.600823 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:34.600784 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" event={"ID":"3bfc6fc4-7482-46af-bf1d-cead4d5104b9","Type":"ContainerStarted","Data":"0551c6cddb50325e0d9ecc18056e09642b44546902fb3a0a8703045aabeabcf7"} Jul 06 22:54:35.707357 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:35.707160 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg"] Jul 06 22:54:35.714463 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:35.714434 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:35.717177 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:35.717143 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kdnx6\"" Jul 06 22:54:35.725735 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:35.725706 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg"] Jul 06 22:54:35.888513 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:35.888478 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q24st\" (UniqueName: \"kubernetes.io/projected/28231327-aed6-419d-a7e1-700a691f5549-kube-api-access-q24st\") pod \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" (UID: \"28231327-aed6-419d-a7e1-700a691f5549\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:35.989973 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:35.989859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24st\" (UniqueName: \"kubernetes.io/projected/28231327-aed6-419d-a7e1-700a691f5549-kube-api-access-q24st\") pod \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" (UID: \"28231327-aed6-419d-a7e1-700a691f5549\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:36.011064 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:36.011030 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q24st\" (UniqueName: \"kubernetes.io/projected/28231327-aed6-419d-a7e1-700a691f5549-kube-api-access-q24st\") pod \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" (UID: \"28231327-aed6-419d-a7e1-700a691f5549\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:36.028817 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:36.028781 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:37.807878 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:37.807843 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg"] Jul 06 22:54:37.809351 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:37.809320 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28231327_aed6_419d_a7e1_700a691f5549.slice/crio-ff044d67333a835a0aa549c9181d0eb77d86c6d705232ce605171cf0cd14990f WatchSource:0}: Error finding container ff044d67333a835a0aa549c9181d0eb77d86c6d705232ce605171cf0cd14990f: Status 404 returned error can't find the container with id ff044d67333a835a0aa549c9181d0eb77d86c6d705232ce605171cf0cd14990f Jul 06 22:54:38.605673 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.605637 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:38.605897 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.605727 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:38.611533 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.611502 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:38.632846 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.632809 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" event={"ID":"3bfc6fc4-7482-46af-bf1d-cead4d5104b9","Type":"ContainerStarted","Data":"d36de18876c0cc6a34ea96834e9e0d76664604e1a0e23539633a952dc45ce0e7"} Jul 06 22:54:38.633031 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.632901 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" Jul 06 22:54:38.634591 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.634562 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" event={"ID":"1c34442a-ed33-41e0-bbff-2bbaade35279","Type":"ContainerStarted","Data":"58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001"} Jul 06 22:54:38.634763 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.634708 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:38.635959 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.635933 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" event={"ID":"28231327-aed6-419d-a7e1-700a691f5549","Type":"ContainerStarted","Data":"ff044d67333a835a0aa549c9181d0eb77d86c6d705232ce605171cf0cd14990f"} Jul 06 22:54:38.640096 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.640073 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7698c68448-dttmw" Jul 06 22:54:38.665903 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.665846 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" podStartSLOduration=1.531802774 podStartE2EDuration="7.665828733s" totalImagesPullingTime="6.134025959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:38.66330009 +0000 UTC m=+491.963001695" watchObservedRunningTime="2026-07-06 22:54:38.665828733 +0000 UTC m=+491.965530335" Jul 06 22:54:38.718409 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.718337 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" podStartSLOduration=2.37965255 podStartE2EDuration="5.718314159s" totalImagesPullingTime="3.338661609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:38.713597463 +0000 UTC m=+492.013299089" watchObservedRunningTime="2026-07-06 22:54:38.718314159 +0000 UTC m=+492.018015762" Jul 06 22:54:38.732359 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:38.732321 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cb959d9b-6qdwz"] Jul 06 22:54:39.642941 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:39.642902 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" event={"ID":"28231327-aed6-419d-a7e1-700a691f5549","Type":"ContainerStarted","Data":"c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2"} Jul 06 22:54:39.643412 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:39.643061 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:39.665878 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:39.665815 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" podStartSLOduration=3.412690478 podStartE2EDuration="4.665798926s" totalImagesPullingTime="1.253108448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:39.664255258 +0000 UTC m=+492.963956860" watchObservedRunningTime="2026-07-06 22:54:39.665798926 +0000 UTC m=+492.965500527" Jul 06 22:54:40.650981 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.650931 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n"] Jul 06 22:54:40.654842 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.654820 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.657348 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.657325 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 22:54:40.657501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.657391 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jmlxb\"" Jul 06 22:54:40.657501 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.657415 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 22:54:40.663373 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.663343 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n"] Jul 06 22:54:40.836573 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.836523 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0910a5f2-2bbb-49d8-8c37-ff472d59504e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.836840 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.836628 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79xcv\" (UniqueName: \"kubernetes.io/projected/0910a5f2-2bbb-49d8-8c37-ff472d59504e-kube-api-access-79xcv\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.836945 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.836917 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0910a5f2-2bbb-49d8-8c37-ff472d59504e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.938228 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.938187 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0910a5f2-2bbb-49d8-8c37-ff472d59504e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.938405 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.938266 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79xcv\" (UniqueName: \"kubernetes.io/projected/0910a5f2-2bbb-49d8-8c37-ff472d59504e-kube-api-access-79xcv\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.938405 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.938320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0910a5f2-2bbb-49d8-8c37-ff472d59504e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.939010 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.938985 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0910a5f2-2bbb-49d8-8c37-ff472d59504e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.940610 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.940586 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0910a5f2-2bbb-49d8-8c37-ff472d59504e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.959358 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.959318 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79xcv\" (UniqueName: \"kubernetes.io/projected/0910a5f2-2bbb-49d8-8c37-ff472d59504e-kube-api-access-79xcv\") pod \"kuadrant-console-plugin-66b5774985-6n45n\" (UID: \"0910a5f2-2bbb-49d8-8c37-ff472d59504e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:40.966051 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:40.966020 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" Jul 06 22:54:41.135323 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:41.135290 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n"] Jul 06 22:54:41.137395 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:54:41.137368 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0910a5f2_2bbb_49d8_8c37_ff472d59504e.slice/crio-fd98dcdb85b5f9b647af5fc4cc859c30db71e823f9596bfe1460b95d4117e7b1 WatchSource:0}: Error finding container fd98dcdb85b5f9b647af5fc4cc859c30db71e823f9596bfe1460b95d4117e7b1: Status 404 returned error can't find the container with id fd98dcdb85b5f9b647af5fc4cc859c30db71e823f9596bfe1460b95d4117e7b1 Jul 06 22:54:41.656113 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:41.656078 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" event={"ID":"0910a5f2-2bbb-49d8-8c37-ff472d59504e","Type":"ContainerStarted","Data":"fd98dcdb85b5f9b647af5fc4cc859c30db71e823f9596bfe1460b95d4117e7b1"} Jul 06 22:54:49.646148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:49.646106 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qhhjt" Jul 06 22:54:49.646627 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:49.646204 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:50.652772 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:50.652737 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:51.822078 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.822037 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9"] Jul 06 22:54:51.822581 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.822329 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" containerName="manager" containerID="cri-o://58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001" gracePeriod=2 Jul 06 22:54:51.836627 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.836586 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9"] Jul 06 22:54:51.872340 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:51.872273 2561 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"1c34442a-ed33-41e0-bbff-2bbaade35279\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\": pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:51.873225 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.873193 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg"] Jul 06 22:54:51.873468 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.873439 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" podUID="28231327-aed6-419d-a7e1-700a691f5549" containerName="manager" containerID="cri-o://c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2" gracePeriod=2 Jul 06 22:54:51.876932 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:51.876854 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:51.879320 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.879248 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg"] Jul 06 22:54:51.882312 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:51.882271 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:51.928400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.927448 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz"] Jul 06 22:54:51.928400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.928040 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28231327-aed6-419d-a7e1-700a691f5549" containerName="manager" Jul 06 22:54:51.928400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.928065 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="28231327-aed6-419d-a7e1-700a691f5549" containerName="manager" Jul 06 22:54:51.928400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.928086 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" containerName="manager" Jul 06 22:54:51.928400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.928095 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" containerName="manager" Jul 06 22:54:51.928400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.928185 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="28231327-aed6-419d-a7e1-700a691f5549" containerName="manager" Jul 06 22:54:51.928400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.928203 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" containerName="manager" Jul 06 22:54:51.933484 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.933449 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc"] Jul 06 22:54:51.933677 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.933544 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:51.937814 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.937779 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" Jul 06 22:54:51.938387 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:51.938354 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:51.951844 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.951794 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc"] Jul 06 22:54:51.954944 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:51.954913 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz"] Jul 06 22:54:51.993204 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:51.993153 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:51.995320 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:51.995283 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:52.044370 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:52.044323 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:52.054309 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.054270 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7r55\" (UniqueName: \"kubernetes.io/projected/55e16c41-d39f-454b-8307-624b4925bdbe-kube-api-access-j7r55\") pod \"limitador-operator-controller-manager-859d7b78bd-g4lpc\" (UID: \"55e16c41-d39f-454b-8307-624b4925bdbe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" Jul 06 22:54:52.054491 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.054347 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nrcfz\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:52.054542 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.054479 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kkrw\" (UniqueName: \"kubernetes.io/projected/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-kube-api-access-2kkrw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nrcfz\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:52.155637 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.155544 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kkrw\" (UniqueName: \"kubernetes.io/projected/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-kube-api-access-2kkrw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nrcfz\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:52.155836 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.155648 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7r55\" (UniqueName: \"kubernetes.io/projected/55e16c41-d39f-454b-8307-624b4925bdbe-kube-api-access-j7r55\") pod \"limitador-operator-controller-manager-859d7b78bd-g4lpc\" (UID: \"55e16c41-d39f-454b-8307-624b4925bdbe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" Jul 06 22:54:52.155836 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.155713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nrcfz\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:52.156149 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.156127 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nrcfz\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:52.186066 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.186027 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7r55\" (UniqueName: \"kubernetes.io/projected/55e16c41-d39f-454b-8307-624b4925bdbe-kube-api-access-j7r55\") pod \"limitador-operator-controller-manager-859d7b78bd-g4lpc\" (UID: \"55e16c41-d39f-454b-8307-624b4925bdbe\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" Jul 06 22:54:52.188062 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.188036 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kkrw\" (UniqueName: \"kubernetes.io/projected/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-kube-api-access-2kkrw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nrcfz\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:52.248449 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.248406 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:54:52.261079 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:52.261037 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" Jul 06 22:54:53.173126 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.173093 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt"] Jul 06 22:54:53.207891 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.207776 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt"] Jul 06 22:54:53.207891 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.207882 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:54:53.210881 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:53.210833 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:54:53.212941 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:54:53.212895 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:54:53.269063 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.269022 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2eafda25-fd81-4f29-b186-942e07b5f7ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9lvtt\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:54:53.269375 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.269338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt64n\" (UniqueName: \"kubernetes.io/projected/2eafda25-fd81-4f29-b186-942e07b5f7ee-kube-api-access-wt64n\") pod \"kuadrant-operator-controller-manager-86878c769f-9lvtt\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:54:53.370586 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.370534 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt64n\" (UniqueName: \"kubernetes.io/projected/2eafda25-fd81-4f29-b186-942e07b5f7ee-kube-api-access-wt64n\") pod \"kuadrant-operator-controller-manager-86878c769f-9lvtt\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:54:53.370810 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.370645 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2eafda25-fd81-4f29-b186-942e07b5f7ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9lvtt\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:54:53.371068 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.371040 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2eafda25-fd81-4f29-b186-942e07b5f7ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9lvtt\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:54:53.382072 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.382035 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt64n\" (UniqueName: \"kubernetes.io/projected/2eafda25-fd81-4f29-b186-942e07b5f7ee-kube-api-access-wt64n\") pod \"kuadrant-operator-controller-manager-86878c769f-9lvtt\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:54:53.521278 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:54:53.521166 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:55:03.311862 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.311646 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:03.317015 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.316922 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:03.325290 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.325212 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:03.330454 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.327785 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:03.330454 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.329744 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:03.330959 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.330657 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt"] Jul 06 22:55:03.335186 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:55:03.335154 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eafda25_fd81_4f29_b186_942e07b5f7ee.slice/crio-5033dca1c80eafc6eef62a28211f4026cdec9e1fae75c92a4a4551da8d885e0b WatchSource:0}: Error finding container 5033dca1c80eafc6eef62a28211f4026cdec9e1fae75c92a4a4551da8d885e0b: Status 404 returned error can't find the container with id 5033dca1c80eafc6eef62a28211f4026cdec9e1fae75c92a4a4551da8d885e0b Jul 06 22:55:03.365713 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.365659 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc"] Jul 06 22:55:03.378028 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:55:03.377989 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e16c41_d39f_454b_8307_624b4925bdbe.slice/crio-f11895170097ec7e2ac699492309dcb5b3f79e0716d9632c55c39570ea0838ac WatchSource:0}: Error finding container f11895170097ec7e2ac699492309dcb5b3f79e0716d9632c55c39570ea0838ac: Status 404 returned error can't find the container with id f11895170097ec7e2ac699492309dcb5b3f79e0716d9632c55c39570ea0838ac Jul 06 22:55:03.479389 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.479343 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q24st\" (UniqueName: \"kubernetes.io/projected/28231327-aed6-419d-a7e1-700a691f5549-kube-api-access-q24st\") pod \"28231327-aed6-419d-a7e1-700a691f5549\" (UID: \"28231327-aed6-419d-a7e1-700a691f5549\") " Jul 06 22:55:03.479572 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.479482 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c34442a-ed33-41e0-bbff-2bbaade35279-extensions-socket-volume\") pod \"1c34442a-ed33-41e0-bbff-2bbaade35279\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " Jul 06 22:55:03.479572 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.479510 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gbz5\" (UniqueName: \"kubernetes.io/projected/1c34442a-ed33-41e0-bbff-2bbaade35279-kube-api-access-5gbz5\") pod \"1c34442a-ed33-41e0-bbff-2bbaade35279\" (UID: \"1c34442a-ed33-41e0-bbff-2bbaade35279\") " Jul 06 22:55:03.480067 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.480036 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c34442a-ed33-41e0-bbff-2bbaade35279-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1c34442a-ed33-41e0-bbff-2bbaade35279" (UID: "1c34442a-ed33-41e0-bbff-2bbaade35279"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:55:03.481652 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.481632 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c34442a-ed33-41e0-bbff-2bbaade35279-kube-api-access-5gbz5" (OuterVolumeSpecName: "kube-api-access-5gbz5") pod "1c34442a-ed33-41e0-bbff-2bbaade35279" (UID: "1c34442a-ed33-41e0-bbff-2bbaade35279"). InnerVolumeSpecName "kube-api-access-5gbz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:55:03.481725 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.481632 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28231327-aed6-419d-a7e1-700a691f5549-kube-api-access-q24st" (OuterVolumeSpecName: "kube-api-access-q24st") pod "28231327-aed6-419d-a7e1-700a691f5549" (UID: "28231327-aed6-419d-a7e1-700a691f5549"). InnerVolumeSpecName "kube-api-access-q24st". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:55:03.568402 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.568375 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz"] Jul 06 22:55:03.570411 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:55:03.570371 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b297ad_0e57_4c7b_aff4_aed2ab796f32.slice/crio-29d753cd7e7eacf5ba0c91dc7987f1dc5877e6503153b6cbd15998c7eabebc70 WatchSource:0}: Error finding container 29d753cd7e7eacf5ba0c91dc7987f1dc5877e6503153b6cbd15998c7eabebc70: Status 404 returned error can't find the container with id 29d753cd7e7eacf5ba0c91dc7987f1dc5877e6503153b6cbd15998c7eabebc70 Jul 06 22:55:03.580493 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.580468 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1c34442a-ed33-41e0-bbff-2bbaade35279-extensions-socket-volume\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:03.580572 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.580496 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gbz5\" (UniqueName: \"kubernetes.io/projected/1c34442a-ed33-41e0-bbff-2bbaade35279-kube-api-access-5gbz5\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:03.580572 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.580506 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q24st\" (UniqueName: \"kubernetes.io/projected/28231327-aed6-419d-a7e1-700a691f5549-kube-api-access-q24st\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:03.759154 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.759102 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6cb959d9b-6qdwz" podUID="c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" containerName="console" containerID="cri-o://732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73" gracePeriod=15 Jul 06 22:55:03.800871 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.800822 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" event={"ID":"0910a5f2-2bbb-49d8-8c37-ff472d59504e","Type":"ContainerStarted","Data":"1f1592f71d56f1b8804e3a99590b3263e02e15c6c5e4545086cdc41353cd4004"} Jul 06 22:55:03.802657 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.802625 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" event={"ID":"f1b297ad-0e57-4c7b-aff4-aed2ab796f32","Type":"ContainerStarted","Data":"f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8"} Jul 06 22:55:03.802867 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.802679 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" event={"ID":"f1b297ad-0e57-4c7b-aff4-aed2ab796f32","Type":"ContainerStarted","Data":"29d753cd7e7eacf5ba0c91dc7987f1dc5877e6503153b6cbd15998c7eabebc70"} Jul 06 22:55:03.802867 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.802735 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:55:03.803372 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.803334 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:03.804047 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.804026 2561 generic.go:356] "Generic (PLEG): container finished" podID="1c34442a-ed33-41e0-bbff-2bbaade35279" containerID="58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001" exitCode=0 Jul 06 22:55:03.804119 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.804070 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:03.804169 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.804117 2561 scope.go:117] "RemoveContainer" containerID="58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001" Jul 06 22:55:03.805193 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.805167 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:03.805570 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.805549 2561 generic.go:356] "Generic (PLEG): container finished" podID="28231327-aed6-419d-a7e1-700a691f5549" containerID="c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2" exitCode=0 Jul 06 22:55:03.805653 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.805593 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:03.807385 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.807353 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" event={"ID":"55e16c41-d39f-454b-8307-624b4925bdbe","Type":"ContainerStarted","Data":"8ae28e26f86ea912ebf58273704729eda92680f4b068b4267044df0e0532845b"} Jul 06 22:55:03.807503 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.807412 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" event={"ID":"55e16c41-d39f-454b-8307-624b4925bdbe","Type":"ContainerStarted","Data":"f11895170097ec7e2ac699492309dcb5b3f79e0716d9632c55c39570ea0838ac"} Jul 06 22:55:03.807503 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.807490 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" Jul 06 22:55:03.809039 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.809017 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" event={"ID":"2eafda25-fd81-4f29-b186-942e07b5f7ee","Type":"ContainerStarted","Data":"d90b52307bcbf5aa820d837c598412ac45f56f90eeba8068b9d8af31fc4bfb6a"} Jul 06 22:55:03.809127 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.809063 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" event={"ID":"2eafda25-fd81-4f29-b186-942e07b5f7ee","Type":"ContainerStarted","Data":"5033dca1c80eafc6eef62a28211f4026cdec9e1fae75c92a4a4551da8d885e0b"} Jul 06 22:55:03.809180 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.809163 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:55:03.826423 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.826366 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6n45n" podStartSLOduration=1.7276916359999999 podStartE2EDuration="23.82634599s" totalImagesPullingTime="22.098654354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:03.824964449 +0000 UTC m=+517.124666052" watchObservedRunningTime="2026-07-06 22:55:03.82634599 +0000 UTC m=+517.126047591" Jul 06 22:55:03.856894 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.856844 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:03.858060 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.858006 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" podStartSLOduration=10.857983048 podStartE2EDuration="10.857983048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:03.854411747 +0000 UTC m=+517.154113383" watchObservedRunningTime="2026-07-06 22:55:03.857983048 +0000 UTC m=+517.157684653" Jul 06 22:55:03.871750 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.871726 2561 scope.go:117] "RemoveContainer" containerID="58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001" Jul 06 22:55:03.872089 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.872063 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001\": container with ID starting with 58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001 not found: ID does not exist" containerID="58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001" Jul 06 22:55:03.872184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.872098 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001"} err="failed to get container status \"58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001\": rpc error: code = NotFound desc = could not find container \"58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001\": container with ID starting with 58d6f47681c443345b9233c2c2964b5d4a97e1bcfdf52147ba6501c3a18bc001 not found: ID does not exist" Jul 06 22:55:03.872184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.872116 2561 scope.go:117] "RemoveContainer" containerID="c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2" Jul 06 22:55:03.882354 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.882291 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" podStartSLOduration=12.882269615 podStartE2EDuration="12.882269615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:03.881766617 +0000 UTC m=+517.181468220" watchObservedRunningTime="2026-07-06 22:55:03.882269615 +0000 UTC m=+517.181971220" Jul 06 22:55:03.883672 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.883645 2561 scope.go:117] "RemoveContainer" containerID="c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2" Jul 06 22:55:03.883985 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.883937 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:03.884106 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.884038 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2\": container with ID starting with c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2 not found: ID does not exist" containerID="c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2" Jul 06 22:55:03.884106 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.884071 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2"} err="failed to get container status \"c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2\": rpc error: code = NotFound desc = could not find container \"c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2\": container with ID starting with c9f19e52de252f1a919f351411e170934b6339fcf1349955832b6550ee2529e2 not found: ID does not exist" Jul 06 22:55:03.910588 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:03.910530 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" podStartSLOduration=12.910513472 podStartE2EDuration="12.910513472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:03.908429304 +0000 UTC m=+517.208130923" watchObservedRunningTime="2026-07-06 22:55:03.910513472 +0000 UTC m=+517.210215073" Jul 06 22:55:03.911904 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.911868 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:03.913850 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:03.913824 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:04.004626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.004596 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:55:04.007067 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:04.007039 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:04.032101 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:04.032058 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:04.085977 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.085942 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-config\") pod \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " Jul 06 22:55:04.085977 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.085992 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-trusted-ca-bundle\") pod \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " Jul 06 22:55:04.086233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086014 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-oauth-serving-cert\") pod \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " Jul 06 22:55:04.086233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086053 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-service-ca\") pod \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " Jul 06 22:55:04.086233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086114 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtsqv\" (UniqueName: \"kubernetes.io/projected/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-kube-api-access-dtsqv\") pod \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " Jul 06 22:55:04.086233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086166 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-serving-cert\") pod \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " Jul 06 22:55:04.086233 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086191 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-oauth-config\") pod \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\" (UID: \"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef\") " Jul 06 22:55:04.086500 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086369 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" (UID: "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:55:04.086500 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086400 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-config" (OuterVolumeSpecName: "console-config") pod "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" (UID: "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:55:04.086500 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086406 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" (UID: "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:55:04.086500 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086487 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:04.086641 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.086487 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" (UID: "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:55:04.088415 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.088383 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" (UID: "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:55:04.088415 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.088409 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-kube-api-access-dtsqv" (OuterVolumeSpecName: "kube-api-access-dtsqv") pod "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" (UID: "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef"). InnerVolumeSpecName "kube-api-access-dtsqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:55:04.088575 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.088411 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" (UID: "c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:55:04.187746 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.187706 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtsqv\" (UniqueName: \"kubernetes.io/projected/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-kube-api-access-dtsqv\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:04.187746 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.187740 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:04.187746 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.187749 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:04.187995 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.187759 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:04.187995 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.187768 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:04.187995 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.187776 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:04.817712 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.817656 2561 generic.go:356] "Generic (PLEG): container finished" podID="c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" containerID="732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73" exitCode=0 Jul 06 22:55:04.818221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.817725 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cb959d9b-6qdwz" Jul 06 22:55:04.818221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.817745 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cb959d9b-6qdwz" event={"ID":"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef","Type":"ContainerDied","Data":"732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73"} Jul 06 22:55:04.818221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.817801 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cb959d9b-6qdwz" event={"ID":"c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef","Type":"ContainerDied","Data":"6806914a13b2f445fbb8ebf2704f608c16e0dcb4c7a7a670f265f908874aadcc"} Jul 06 22:55:04.818221 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.817827 2561 scope.go:117] "RemoveContainer" containerID="732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73" Jul 06 22:55:04.820993 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:04.820961 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:04.827343 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:04.827304 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:04.827937 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.827917 2561 scope.go:117] "RemoveContainer" containerID="732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73" Jul 06 22:55:04.828252 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:04.828229 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73\": container with ID starting with 732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73 not found: ID does not exist" containerID="732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73" Jul 06 22:55:04.828304 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.828263 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73"} err="failed to get container status \"732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73\": rpc error: code = NotFound desc = could not find container \"732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73\": container with ID starting with 732d986d73ac2c2f5c1272f870a9822e325f1bc723c7b533e44da64b80947d73 not found: ID does not exist" Jul 06 22:55:04.829787 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:04.829763 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rg8qg\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="28231327-aed6-419d-a7e1-700a691f5549" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rg8qg" Jul 06 22:55:04.848867 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.848827 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cb959d9b-6qdwz"] Jul 06 22:55:04.854022 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:04.853977 2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v8zq9\" is forbidden: User \"system:node:ip-10-0-128-38.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-38.ec2.internal' and this object" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v8zq9" Jul 06 22:55:04.854441 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:04.854424 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cb959d9b-6qdwz"] Jul 06 22:55:05.220337 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:05.220303 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c34442a-ed33-41e0-bbff-2bbaade35279" path="/var/lib/kubelet/pods/1c34442a-ed33-41e0-bbff-2bbaade35279/volumes" Jul 06 22:55:05.220640 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:05.220627 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28231327-aed6-419d-a7e1-700a691f5549" path="/var/lib/kubelet/pods/28231327-aed6-419d-a7e1-700a691f5549/volumes" Jul 06 22:55:05.221017 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:05.221003 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" path="/var/lib/kubelet/pods/c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef/volumes" Jul 06 22:55:14.820790 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:14.820753 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 22:55:14.821386 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:14.820840 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4lpc" Jul 06 22:55:14.821386 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:14.821101 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:55:14.883148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:14.883099 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz"] Jul 06 22:55:14.883382 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:14.883351 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" podUID="f1b297ad-0e57-4c7b-aff4-aed2ab796f32" containerName="manager" containerID="cri-o://f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8" gracePeriod=10 Jul 06 22:55:15.152433 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.152406 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:55:15.189012 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.188978 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kkrw\" (UniqueName: \"kubernetes.io/projected/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-kube-api-access-2kkrw\") pod \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " Jul 06 22:55:15.189185 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.189075 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-extensions-socket-volume\") pod \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\" (UID: \"f1b297ad-0e57-4c7b-aff4-aed2ab796f32\") " Jul 06 22:55:15.189443 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.189415 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f1b297ad-0e57-4c7b-aff4-aed2ab796f32" (UID: "f1b297ad-0e57-4c7b-aff4-aed2ab796f32"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:55:15.191186 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.191161 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-kube-api-access-2kkrw" (OuterVolumeSpecName: "kube-api-access-2kkrw") pod "f1b297ad-0e57-4c7b-aff4-aed2ab796f32" (UID: "f1b297ad-0e57-4c7b-aff4-aed2ab796f32"). InnerVolumeSpecName "kube-api-access-2kkrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:55:15.290695 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.290657 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kkrw\" (UniqueName: \"kubernetes.io/projected/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-kube-api-access-2kkrw\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:15.290869 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.290705 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b297ad-0e57-4c7b-aff4-aed2ab796f32-extensions-socket-volume\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:15.883679 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.883641 2561 generic.go:356] "Generic (PLEG): container finished" podID="f1b297ad-0e57-4c7b-aff4-aed2ab796f32" containerID="f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8" exitCode=0 Jul 06 22:55:15.884090 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.883701 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" Jul 06 22:55:15.884090 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.883734 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" event={"ID":"f1b297ad-0e57-4c7b-aff4-aed2ab796f32","Type":"ContainerDied","Data":"f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8"} Jul 06 22:55:15.884090 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.883791 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz" event={"ID":"f1b297ad-0e57-4c7b-aff4-aed2ab796f32","Type":"ContainerDied","Data":"29d753cd7e7eacf5ba0c91dc7987f1dc5877e6503153b6cbd15998c7eabebc70"} Jul 06 22:55:15.884090 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.883814 2561 scope.go:117] "RemoveContainer" containerID="f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8" Jul 06 22:55:15.893070 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.893051 2561 scope.go:117] "RemoveContainer" containerID="f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8" Jul 06 22:55:15.893359 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:15.893336 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8\": container with ID starting with f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8 not found: ID does not exist" containerID="f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8" Jul 06 22:55:15.893406 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.893369 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8"} err="failed to get container status \"f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8\": rpc error: code = NotFound desc = could not find container \"f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8\": container with ID starting with f2b9fe164666eac1a9d0ed5328c65542dca39368b017a42ac76dae343da659f8 not found: ID does not exist" Jul 06 22:55:15.906422 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.906382 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz"] Jul 06 22:55:15.916437 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:15.916407 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nrcfz"] Jul 06 22:55:17.220172 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:17.220138 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1b297ad-0e57-4c7b-aff4-aed2ab796f32" path="/var/lib/kubelet/pods/f1b297ad-0e57-4c7b-aff4-aed2ab796f32/volumes" Jul 06 22:55:31.170277 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.170232 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn"] Jul 06 22:55:31.170802 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.170786 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1b297ad-0e57-4c7b-aff4-aed2ab796f32" containerName="manager" Jul 06 22:55:31.170855 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.170805 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1b297ad-0e57-4c7b-aff4-aed2ab796f32" containerName="manager" Jul 06 22:55:31.170855 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.170822 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" containerName="console" Jul 06 22:55:31.170855 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.170830 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" containerName="console" Jul 06 22:55:31.170952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.170936 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1b297ad-0e57-4c7b-aff4-aed2ab796f32" containerName="manager" Jul 06 22:55:31.170952 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.170948 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d2a0ee-ac17-4bfd-98ed-6ee78db2e0ef" containerName="console" Jul 06 22:55:31.179439 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.179406 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.184548 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.184518 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn"] Jul 06 22:55:31.185201 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.185178 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nxvkb\"" Jul 06 22:55:31.245604 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245564 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.245829 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fd0385-52bf-4df5-afec-e6405b79461e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.245829 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245655 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.245829 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245706 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fd0385-52bf-4df5-afec-e6405b79461e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.245829 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245749 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.245829 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245771 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrtsh\" (UniqueName: \"kubernetes.io/projected/61fd0385-52bf-4df5-afec-e6405b79461e-kube-api-access-hrtsh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.245829 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245815 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.246032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fd0385-52bf-4df5-afec-e6405b79461e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.246032 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.245868 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347056 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347317 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347130 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fd0385-52bf-4df5-afec-e6405b79461e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347317 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347164 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347317 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347189 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fd0385-52bf-4df5-afec-e6405b79461e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347317 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347317 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347265 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrtsh\" (UniqueName: \"kubernetes.io/projected/61fd0385-52bf-4df5-afec-e6405b79461e-kube-api-access-hrtsh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347323 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347354 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fd0385-52bf-4df5-afec-e6405b79461e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347526 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347606 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347548 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347971 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347666 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347971 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347790 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.347971 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.347956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61fd0385-52bf-4df5-afec-e6405b79461e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.349680 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.349657 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61fd0385-52bf-4df5-afec-e6405b79461e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.350022 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.349949 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61fd0385-52bf-4df5-afec-e6405b79461e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.356288 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.356253 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61fd0385-52bf-4df5-afec-e6405b79461e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.356434 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.356385 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrtsh\" (UniqueName: \"kubernetes.io/projected/61fd0385-52bf-4df5-afec-e6405b79461e-kube-api-access-hrtsh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hqqzn\" (UID: \"61fd0385-52bf-4df5-afec-e6405b79461e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.491594 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.491500 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:31.639565 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.639532 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn"] Jul 06 22:55:31.641063 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:55:31.641032 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fd0385_52bf_4df5_afec_e6405b79461e.slice/crio-3d826435bb843d1c8ba5ff37fe5da75c7de3bfe46a04144d67497024dd87fc59 WatchSource:0}: Error finding container 3d826435bb843d1c8ba5ff37fe5da75c7de3bfe46a04144d67497024dd87fc59: Status 404 returned error can't find the container with id 3d826435bb843d1c8ba5ff37fe5da75c7de3bfe46a04144d67497024dd87fc59 Jul 06 22:55:31.643323 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.643280 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:55:31.643413 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.643364 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:55:31.643413 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.643397 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:55:31.974094 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.974049 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" event={"ID":"61fd0385-52bf-4df5-afec-e6405b79461e","Type":"ContainerStarted","Data":"e2c11160c3d3a2a1b7688000163c6b71efcaa347ef45c2958266cc3176681fae"} Jul 06 22:55:31.974469 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.974116 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" event={"ID":"61fd0385-52bf-4df5-afec-e6405b79461e","Type":"ContainerStarted","Data":"3d826435bb843d1c8ba5ff37fe5da75c7de3bfe46a04144d67497024dd87fc59"} Jul 06 22:55:31.994244 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:31.994189 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" podStartSLOduration=0.994168252 podStartE2EDuration="994.168252ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:31.992540861 +0000 UTC m=+545.292242487" watchObservedRunningTime="2026-07-06 22:55:31.994168252 +0000 UTC m=+545.293869853" Jul 06 22:55:32.491884 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:32.491847 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:32.496866 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:32.496834 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:32.980229 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:32.980198 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:32.981406 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:32.981376 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:55:33.734226 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:33.734183 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xr7fj"] Jul 06 22:55:33.737932 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:33.737907 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xr7fj" Jul 06 22:55:33.740761 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:33.740731 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zg7r9\"" Jul 06 22:55:33.746838 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:33.746810 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xr7fj"] Jul 06 22:55:33.872546 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:33.872501 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrvf9\" (UniqueName: \"kubernetes.io/projected/fd6abe77-632d-4827-94bf-cc5141ff7a68-kube-api-access-nrvf9\") pod \"authorino-58cd955675-xr7fj\" (UID: \"fd6abe77-632d-4827-94bf-cc5141ff7a68\") " pod="kuadrant-system/authorino-58cd955675-xr7fj" Jul 06 22:55:33.973921 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:33.973875 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrvf9\" (UniqueName: \"kubernetes.io/projected/fd6abe77-632d-4827-94bf-cc5141ff7a68-kube-api-access-nrvf9\") pod \"authorino-58cd955675-xr7fj\" (UID: \"fd6abe77-632d-4827-94bf-cc5141ff7a68\") " pod="kuadrant-system/authorino-58cd955675-xr7fj" Jul 06 22:55:34.007393 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.007312 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrvf9\" (UniqueName: \"kubernetes.io/projected/fd6abe77-632d-4827-94bf-cc5141ff7a68-kube-api-access-nrvf9\") pod \"authorino-58cd955675-xr7fj\" (UID: \"fd6abe77-632d-4827-94bf-cc5141ff7a68\") " pod="kuadrant-system/authorino-58cd955675-xr7fj" Jul 06 22:55:34.022000 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.021962 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-dmrqp"] Jul 06 22:55:34.025545 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.025525 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dmrqp" Jul 06 22:55:34.046048 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.046016 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dmrqp"] Jul 06 22:55:34.049723 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.049674 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xr7fj" Jul 06 22:55:34.175482 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.175441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gdhz\" (UniqueName: \"kubernetes.io/projected/89112e4e-dda1-4261-aa48-ac89a59c167c-kube-api-access-5gdhz\") pod \"authorino-75d556bd77-dmrqp\" (UID: \"89112e4e-dda1-4261-aa48-ac89a59c167c\") " pod="kuadrant-system/authorino-75d556bd77-dmrqp" Jul 06 22:55:34.193172 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.193142 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xr7fj"] Jul 06 22:55:34.195070 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:55:34.195016 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd6abe77_632d_4827_94bf_cc5141ff7a68.slice/crio-41199eca5ee8c8e08e689e1eefa996e928a38d97169d32c2e5162634a8aed85b WatchSource:0}: Error finding container 41199eca5ee8c8e08e689e1eefa996e928a38d97169d32c2e5162634a8aed85b: Status 404 returned error can't find the container with id 41199eca5ee8c8e08e689e1eefa996e928a38d97169d32c2e5162634a8aed85b Jul 06 22:55:34.276324 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.276227 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gdhz\" (UniqueName: \"kubernetes.io/projected/89112e4e-dda1-4261-aa48-ac89a59c167c-kube-api-access-5gdhz\") pod \"authorino-75d556bd77-dmrqp\" (UID: \"89112e4e-dda1-4261-aa48-ac89a59c167c\") " pod="kuadrant-system/authorino-75d556bd77-dmrqp" Jul 06 22:55:34.284203 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.284172 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gdhz\" (UniqueName: \"kubernetes.io/projected/89112e4e-dda1-4261-aa48-ac89a59c167c-kube-api-access-5gdhz\") pod \"authorino-75d556bd77-dmrqp\" (UID: \"89112e4e-dda1-4261-aa48-ac89a59c167c\") " pod="kuadrant-system/authorino-75d556bd77-dmrqp" Jul 06 22:55:34.335178 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.335136 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dmrqp" Jul 06 22:55:34.479515 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.479484 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dmrqp"] Jul 06 22:55:34.481492 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:55:34.481461 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89112e4e_dda1_4261_aa48_ac89a59c167c.slice/crio-fd73612aae81106e15f42ede1ebc46b29c34d2df4c19d517339d949a2b8210f3 WatchSource:0}: Error finding container fd73612aae81106e15f42ede1ebc46b29c34d2df4c19d517339d949a2b8210f3: Status 404 returned error can't find the container with id fd73612aae81106e15f42ede1ebc46b29c34d2df4c19d517339d949a2b8210f3 Jul 06 22:55:34.997962 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:34.997874 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dmrqp" event={"ID":"89112e4e-dda1-4261-aa48-ac89a59c167c","Type":"ContainerStarted","Data":"fd73612aae81106e15f42ede1ebc46b29c34d2df4c19d517339d949a2b8210f3"} Jul 06 22:55:35.001591 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:35.001550 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xr7fj" event={"ID":"fd6abe77-632d-4827-94bf-cc5141ff7a68","Type":"ContainerStarted","Data":"41199eca5ee8c8e08e689e1eefa996e928a38d97169d32c2e5162634a8aed85b"} Jul 06 22:55:38.022754 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:38.022707 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dmrqp" event={"ID":"89112e4e-dda1-4261-aa48-ac89a59c167c","Type":"ContainerStarted","Data":"c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28"} Jul 06 22:55:38.024252 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:38.024221 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xr7fj" event={"ID":"fd6abe77-632d-4827-94bf-cc5141ff7a68","Type":"ContainerStarted","Data":"b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233"} Jul 06 22:55:38.036792 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:38.036674 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-dmrqp" podStartSLOduration=1.722722068 podStartE2EDuration="5.036655526s" totalImagesPullingTime="3.313933458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:38.035965723 +0000 UTC m=+551.335667325" watchObservedRunningTime="2026-07-06 22:55:38.036655526 +0000 UTC m=+551.336357133" Jul 06 22:55:38.051189 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:38.051132 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xr7fj" podStartSLOduration=1.464764998 podStartE2EDuration="5.051114358s" totalImagesPullingTime="3.58634936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:55:38.048910783 +0000 UTC m=+551.348612388" watchObservedRunningTime="2026-07-06 22:55:38.051114358 +0000 UTC m=+551.350815962" Jul 06 22:55:38.077359 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:38.077318 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xr7fj"] Jul 06 22:55:40.035625 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:40.035579 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xr7fj" podUID="fd6abe77-632d-4827-94bf-cc5141ff7a68" containerName="authorino" containerID="cri-o://b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233" gracePeriod=30 Jul 06 22:55:40.295490 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:40.295410 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xr7fj" Jul 06 22:55:40.439648 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:40.439608 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrvf9\" (UniqueName: \"kubernetes.io/projected/fd6abe77-632d-4827-94bf-cc5141ff7a68-kube-api-access-nrvf9\") pod \"fd6abe77-632d-4827-94bf-cc5141ff7a68\" (UID: \"fd6abe77-632d-4827-94bf-cc5141ff7a68\") " Jul 06 22:55:40.441715 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:40.441674 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6abe77-632d-4827-94bf-cc5141ff7a68-kube-api-access-nrvf9" (OuterVolumeSpecName: "kube-api-access-nrvf9") pod "fd6abe77-632d-4827-94bf-cc5141ff7a68" (UID: "fd6abe77-632d-4827-94bf-cc5141ff7a68"). InnerVolumeSpecName "kube-api-access-nrvf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:55:40.540801 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:40.540768 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrvf9\" (UniqueName: \"kubernetes.io/projected/fd6abe77-632d-4827-94bf-cc5141ff7a68-kube-api-access-nrvf9\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:55:41.042306 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.042263 2561 generic.go:356] "Generic (PLEG): container finished" podID="fd6abe77-632d-4827-94bf-cc5141ff7a68" containerID="b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233" exitCode=0 Jul 06 22:55:41.042306 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.042310 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xr7fj" Jul 06 22:55:41.042797 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.042346 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xr7fj" event={"ID":"fd6abe77-632d-4827-94bf-cc5141ff7a68","Type":"ContainerDied","Data":"b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233"} Jul 06 22:55:41.042797 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.042395 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xr7fj" event={"ID":"fd6abe77-632d-4827-94bf-cc5141ff7a68","Type":"ContainerDied","Data":"41199eca5ee8c8e08e689e1eefa996e928a38d97169d32c2e5162634a8aed85b"} Jul 06 22:55:41.042797 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.042413 2561 scope.go:117] "RemoveContainer" containerID="b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233" Jul 06 22:55:41.051413 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.051394 2561 scope.go:117] "RemoveContainer" containerID="b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233" Jul 06 22:55:41.051697 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:55:41.051667 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233\": container with ID starting with b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233 not found: ID does not exist" containerID="b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233" Jul 06 22:55:41.051757 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.051714 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233"} err="failed to get container status \"b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233\": rpc error: code = NotFound desc = could not find container \"b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233\": container with ID starting with b0a904c17bd28df7092c8468b25a35afa8fdd0256a9be8e0049367603070d233 not found: ID does not exist" Jul 06 22:55:41.063347 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.063314 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xr7fj"] Jul 06 22:55:41.065568 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.065544 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xr7fj"] Jul 06 22:55:41.220281 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:55:41.220248 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd6abe77-632d-4827-94bf-cc5141ff7a68" path="/var/lib/kubelet/pods/fd6abe77-632d-4827-94bf-cc5141ff7a68/volumes" Jul 06 22:56:01.862630 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:01.862581 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2ln"] Jul 06 22:56:01.863592 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:01.863563 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd6abe77-632d-4827-94bf-cc5141ff7a68" containerName="authorino" Jul 06 22:56:01.863592 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:01.863595 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd6abe77-632d-4827-94bf-cc5141ff7a68" containerName="authorino" Jul 06 22:56:01.863810 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:01.863795 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd6abe77-632d-4827-94bf-cc5141ff7a68" containerName="authorino" Jul 06 22:56:01.867876 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:01.867843 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" Jul 06 22:56:01.873656 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:01.873617 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2ln"] Jul 06 22:56:01.913346 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:01.913303 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pds8h\" (UniqueName: \"kubernetes.io/projected/55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b-kube-api-access-pds8h\") pod \"authorino-5d4b67c446-gs2ln\" (UID: \"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b\") " pod="kuadrant-system/authorino-5d4b67c446-gs2ln" Jul 06 22:56:02.013906 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.013849 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pds8h\" (UniqueName: \"kubernetes.io/projected/55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b-kube-api-access-pds8h\") pod \"authorino-5d4b67c446-gs2ln\" (UID: \"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b\") " pod="kuadrant-system/authorino-5d4b67c446-gs2ln" Jul 06 22:56:02.022264 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.022233 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pds8h\" (UniqueName: \"kubernetes.io/projected/55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b-kube-api-access-pds8h\") pod \"authorino-5d4b67c446-gs2ln\" (UID: \"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b\") " pod="kuadrant-system/authorino-5d4b67c446-gs2ln" Jul 06 22:56:02.058079 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.058033 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2ln"] Jul 06 22:56:02.058334 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.058321 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" Jul 06 22:56:02.231528 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.231462 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2ln"] Jul 06 22:56:02.233844 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:56:02.233812 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55284a4d_c8f5_49e6_bf8d_d9a1a832bf5b.slice/crio-df8137404c37649edbf5b5ae38dc8311d82462501ec30d0db54a16a3a50ab9bb WatchSource:0}: Error finding container df8137404c37649edbf5b5ae38dc8311d82462501ec30d0db54a16a3a50ab9bb: Status 404 returned error can't find the container with id df8137404c37649edbf5b5ae38dc8311d82462501ec30d0db54a16a3a50ab9bb Jul 06 22:56:02.297839 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.297799 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9d86bdb89-6qmj5"] Jul 06 22:56:02.302489 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.302463 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.305103 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.305079 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 22:56:02.310569 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.310536 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9d86bdb89-6qmj5"] Jul 06 22:56:02.321944 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.321906 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3fd389e3-bac3-4b02-9155-9b5a91d258fd-tls-cert\") pod \"authorino-9d86bdb89-6qmj5\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.322136 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.321953 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh6l8\" (UniqueName: \"kubernetes.io/projected/3fd389e3-bac3-4b02-9155-9b5a91d258fd-kube-api-access-kh6l8\") pod \"authorino-9d86bdb89-6qmj5\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.423269 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.423237 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3fd389e3-bac3-4b02-9155-9b5a91d258fd-tls-cert\") pod \"authorino-9d86bdb89-6qmj5\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.423479 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.423278 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh6l8\" (UniqueName: \"kubernetes.io/projected/3fd389e3-bac3-4b02-9155-9b5a91d258fd-kube-api-access-kh6l8\") pod \"authorino-9d86bdb89-6qmj5\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.425873 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.425844 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3fd389e3-bac3-4b02-9155-9b5a91d258fd-tls-cert\") pod \"authorino-9d86bdb89-6qmj5\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.431384 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.431351 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh6l8\" (UniqueName: \"kubernetes.io/projected/3fd389e3-bac3-4b02-9155-9b5a91d258fd-kube-api-access-kh6l8\") pod \"authorino-9d86bdb89-6qmj5\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.614011 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.613976 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:56:02.769184 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:02.769150 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9d86bdb89-6qmj5"] Jul 06 22:56:02.772152 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:56:02.772057 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fd389e3_bac3_4b02_9155_9b5a91d258fd.slice/crio-5d38de68fb44cb1be54095f666d3f45a42bc65003d3c2e7f422f11491eb614d3 WatchSource:0}: Error finding container 5d38de68fb44cb1be54095f666d3f45a42bc65003d3c2e7f422f11491eb614d3: Status 404 returned error can't find the container with id 5d38de68fb44cb1be54095f666d3f45a42bc65003d3c2e7f422f11491eb614d3 Jul 06 22:56:03.173013 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.172981 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" event={"ID":"3fd389e3-bac3-4b02-9155-9b5a91d258fd","Type":"ContainerStarted","Data":"5d38de68fb44cb1be54095f666d3f45a42bc65003d3c2e7f422f11491eb614d3"} Jul 06 22:56:03.174796 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.174757 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" event={"ID":"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b","Type":"ContainerStarted","Data":"3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b"} Jul 06 22:56:03.174942 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.174808 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" event={"ID":"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b","Type":"ContainerStarted","Data":"df8137404c37649edbf5b5ae38dc8311d82462501ec30d0db54a16a3a50ab9bb"} Jul 06 22:56:03.174942 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.174836 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" podUID="55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b" containerName="authorino" containerID="cri-o://3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b" gracePeriod=30 Jul 06 22:56:03.189418 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.189369 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" podStartSLOduration=1.796665416 podStartE2EDuration="2.189351929s" totalImagesPullingTime="392.686513ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:03.188677612 +0000 UTC m=+576.488379216" watchObservedRunningTime="2026-07-06 22:56:03.189351929 +0000 UTC m=+576.489053531" Jul 06 22:56:03.424332 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.424295 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" Jul 06 22:56:03.431289 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.431260 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pds8h\" (UniqueName: \"kubernetes.io/projected/55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b-kube-api-access-pds8h\") pod \"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b\" (UID: \"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b\") " Jul 06 22:56:03.433291 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.433264 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b-kube-api-access-pds8h" (OuterVolumeSpecName: "kube-api-access-pds8h") pod "55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b" (UID: "55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b"). InnerVolumeSpecName "kube-api-access-pds8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:56:03.531919 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:03.531875 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pds8h\" (UniqueName: \"kubernetes.io/projected/55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b-kube-api-access-pds8h\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:56:04.184617 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.184578 2561 generic.go:356] "Generic (PLEG): container finished" podID="55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b" containerID="3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b" exitCode=0 Jul 06 22:56:04.184617 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.184618 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" Jul 06 22:56:04.185146 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.184655 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" event={"ID":"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b","Type":"ContainerDied","Data":"3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b"} Jul 06 22:56:04.185146 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.184724 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2ln" event={"ID":"55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b","Type":"ContainerDied","Data":"df8137404c37649edbf5b5ae38dc8311d82462501ec30d0db54a16a3a50ab9bb"} Jul 06 22:56:04.185146 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.184746 2561 scope.go:117] "RemoveContainer" containerID="3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b" Jul 06 22:56:04.186327 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.186305 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" event={"ID":"3fd389e3-bac3-4b02-9155-9b5a91d258fd","Type":"ContainerStarted","Data":"0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2"} Jul 06 22:56:04.193658 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.193640 2561 scope.go:117] "RemoveContainer" containerID="3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b" Jul 06 22:56:04.194042 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:56:04.194021 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b\": container with ID starting with 3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b not found: ID does not exist" containerID="3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b" Jul 06 22:56:04.194132 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.194051 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b"} err="failed to get container status \"3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b\": rpc error: code = NotFound desc = could not find container \"3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b\": container with ID starting with 3aa4bdc70481eac438517be0c2a920682459084024b5d7fc5e6aa4141a7bc01b not found: ID does not exist" Jul 06 22:56:04.203098 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.203042 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" podStartSLOduration=1.823740967 podStartE2EDuration="2.203026053s" totalImagesPullingTime="379.285086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:04.201715485 +0000 UTC m=+577.501417078" watchObservedRunningTime="2026-07-06 22:56:04.203026053 +0000 UTC m=+577.502727660" Jul 06 22:56:04.217046 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.217009 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2ln"] Jul 06 22:56:04.219700 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.219653 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2ln"] Jul 06 22:56:04.227895 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.227803 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dmrqp"] Jul 06 22:56:04.228080 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.228004 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-dmrqp" podUID="89112e4e-dda1-4261-aa48-ac89a59c167c" containerName="authorino" containerID="cri-o://c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28" gracePeriod=30 Jul 06 22:56:04.494608 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.494579 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dmrqp" Jul 06 22:56:04.539772 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.539736 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gdhz\" (UniqueName: \"kubernetes.io/projected/89112e4e-dda1-4261-aa48-ac89a59c167c-kube-api-access-5gdhz\") pod \"89112e4e-dda1-4261-aa48-ac89a59c167c\" (UID: \"89112e4e-dda1-4261-aa48-ac89a59c167c\") " Jul 06 22:56:04.541966 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.541921 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89112e4e-dda1-4261-aa48-ac89a59c167c-kube-api-access-5gdhz" (OuterVolumeSpecName: "kube-api-access-5gdhz") pod "89112e4e-dda1-4261-aa48-ac89a59c167c" (UID: "89112e4e-dda1-4261-aa48-ac89a59c167c"). InnerVolumeSpecName "kube-api-access-5gdhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:56:04.640724 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:04.640599 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gdhz\" (UniqueName: \"kubernetes.io/projected/89112e4e-dda1-4261-aa48-ac89a59c167c-kube-api-access-5gdhz\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:56:05.193095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.193061 2561 generic.go:356] "Generic (PLEG): container finished" podID="89112e4e-dda1-4261-aa48-ac89a59c167c" containerID="c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28" exitCode=0 Jul 06 22:56:05.193626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.193111 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dmrqp" Jul 06 22:56:05.193626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.193143 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dmrqp" event={"ID":"89112e4e-dda1-4261-aa48-ac89a59c167c","Type":"ContainerDied","Data":"c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28"} Jul 06 22:56:05.193626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.193181 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dmrqp" event={"ID":"89112e4e-dda1-4261-aa48-ac89a59c167c","Type":"ContainerDied","Data":"fd73612aae81106e15f42ede1ebc46b29c34d2df4c19d517339d949a2b8210f3"} Jul 06 22:56:05.193626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.193203 2561 scope.go:117] "RemoveContainer" containerID="c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28" Jul 06 22:56:05.202489 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.202471 2561 scope.go:117] "RemoveContainer" containerID="c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28" Jul 06 22:56:05.202904 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:56:05.202886 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28\": container with ID starting with c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28 not found: ID does not exist" containerID="c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28" Jul 06 22:56:05.202950 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.202912 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28"} err="failed to get container status \"c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28\": rpc error: code = NotFound desc = could not find container \"c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28\": container with ID starting with c5b7eaf655b4ccd6f14543318300fd63023ed5e6db5b1dd450106ece92ca5b28 not found: ID does not exist" Jul 06 22:56:05.213853 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.213821 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dmrqp"] Jul 06 22:56:05.226575 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.226542 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b" path="/var/lib/kubelet/pods/55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b/volumes" Jul 06 22:56:05.226862 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:05.226847 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dmrqp"] Jul 06 22:56:07.220051 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:07.220009 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89112e4e-dda1-4261-aa48-ac89a59c167c" path="/var/lib/kubelet/pods/89112e4e-dda1-4261-aa48-ac89a59c167c/volumes" Jul 06 22:56:27.128808 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:27.128778 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 22:56:27.133673 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:27.133648 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 22:56:27.134865 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:27.134840 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:56:27.146038 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:27.146009 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 22:56:43.974473 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.974435 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-b5tlr"] Jul 06 22:56:43.977218 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.974843 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89112e4e-dda1-4261-aa48-ac89a59c167c" containerName="authorino" Jul 06 22:56:43.977218 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.974855 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="89112e4e-dda1-4261-aa48-ac89a59c167c" containerName="authorino" Jul 06 22:56:43.977218 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.974876 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b" containerName="authorino" Jul 06 22:56:43.977218 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.974882 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b" containerName="authorino" Jul 06 22:56:43.977218 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.974949 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="89112e4e-dda1-4261-aa48-ac89a59c167c" containerName="authorino" Jul 06 22:56:43.977218 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.974959 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="55284a4d-c8f5-49e6-bf8d-d9a1a832bf5b" containerName="authorino" Jul 06 22:56:43.978008 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.977987 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:43.980729 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.980701 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nrgpx\"" Jul 06 22:56:43.980885 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.980679 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 22:56:43.987739 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:43.987674 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-b5tlr"] Jul 06 22:56:44.106660 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.106619 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d3caabc-69e1-4913-94b9-c9ca44a92f3a-plugins-config-volume\") pod \"payload-processing-58f86c7798-b5tlr\" (UID: \"3d3caabc-69e1-4913-94b9-c9ca44a92f3a\") " pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:44.106660 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.106662 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmlvc\" (UniqueName: \"kubernetes.io/projected/3d3caabc-69e1-4913-94b9-c9ca44a92f3a-kube-api-access-fmlvc\") pod \"payload-processing-58f86c7798-b5tlr\" (UID: \"3d3caabc-69e1-4913-94b9-c9ca44a92f3a\") " pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:44.207979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.207936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d3caabc-69e1-4913-94b9-c9ca44a92f3a-plugins-config-volume\") pod \"payload-processing-58f86c7798-b5tlr\" (UID: \"3d3caabc-69e1-4913-94b9-c9ca44a92f3a\") " pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:44.207979 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.207980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmlvc\" (UniqueName: \"kubernetes.io/projected/3d3caabc-69e1-4913-94b9-c9ca44a92f3a-kube-api-access-fmlvc\") pod \"payload-processing-58f86c7798-b5tlr\" (UID: \"3d3caabc-69e1-4913-94b9-c9ca44a92f3a\") " pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:44.208774 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.208754 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d3caabc-69e1-4913-94b9-c9ca44a92f3a-plugins-config-volume\") pod \"payload-processing-58f86c7798-b5tlr\" (UID: \"3d3caabc-69e1-4913-94b9-c9ca44a92f3a\") " pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:44.216422 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.216399 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmlvc\" (UniqueName: \"kubernetes.io/projected/3d3caabc-69e1-4913-94b9-c9ca44a92f3a-kube-api-access-fmlvc\") pod \"payload-processing-58f86c7798-b5tlr\" (UID: \"3d3caabc-69e1-4913-94b9-c9ca44a92f3a\") " pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:44.291400 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.291293 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:44.438372 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.438336 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-b5tlr"] Jul 06 22:56:44.441726 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:56:44.441666 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3caabc_69e1_4913_94b9_c9ca44a92f3a.slice/crio-b0c8d592666d413829ee9c5d837bf34da019cbe79a1b923c4876f31075c4ba66 WatchSource:0}: Error finding container b0c8d592666d413829ee9c5d837bf34da019cbe79a1b923c4876f31075c4ba66: Status 404 returned error can't find the container with id b0c8d592666d413829ee9c5d837bf34da019cbe79a1b923c4876f31075c4ba66 Jul 06 22:56:44.443840 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:44.443820 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:56:45.449793 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:45.449708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" event={"ID":"3d3caabc-69e1-4913-94b9-c9ca44a92f3a","Type":"ContainerStarted","Data":"b0c8d592666d413829ee9c5d837bf34da019cbe79a1b923c4876f31075c4ba66"} Jul 06 22:56:46.458536 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:46.458491 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" event={"ID":"3d3caabc-69e1-4913-94b9-c9ca44a92f3a","Type":"ContainerStarted","Data":"cb8381e93fca5f1060a299f3deb045619503e4c84ec4507492abba188c586fd5"} Jul 06 22:56:46.458959 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:46.458560 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:46.475782 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:46.475731 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" podStartSLOduration=2.026021001 podStartE2EDuration="3.475713864s" totalImagesPullingTime="1.449692863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:46.474420177 +0000 UTC m=+619.774121780" watchObservedRunningTime="2026-07-06 22:56:46.475713864 +0000 UTC m=+619.775415463" Jul 06 22:56:52.492954 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:52.492905 2561 generic.go:356] "Generic (PLEG): container finished" podID="61fd0385-52bf-4df5-afec-e6405b79461e" containerID="e2c11160c3d3a2a1b7688000163c6b71efcaa347ef45c2958266cc3176681fae" exitCode=0 Jul 06 22:56:52.493408 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:52.492971 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" event={"ID":"61fd0385-52bf-4df5-afec-e6405b79461e","Type":"ContainerDied","Data":"e2c11160c3d3a2a1b7688000163c6b71efcaa347ef45c2958266cc3176681fae"} Jul 06 22:56:52.493479 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:52.493426 2561 scope.go:117] "RemoveContainer" containerID="e2c11160c3d3a2a1b7688000163c6b71efcaa347ef45c2958266cc3176681fae" Jul 06 22:56:52.494470 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:52.494439 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:56:52.494560 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:52.494525 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:56:52.494602 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:52.494561 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 22:56:53.500175 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:53.500138 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" event={"ID":"61fd0385-52bf-4df5-afec-e6405b79461e","Type":"ContainerStarted","Data":"8b76141df8a078714e42d53a2748a2731c2928de7d2838116ea6dac2a02a00f3"} Jul 06 22:56:54.492369 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:54.492331 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:56:54.492587 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:54.492563 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hqqzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 06 22:56:54.492637 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:54.492615 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" podUID="61fd0385-52bf-4df5-afec-e6405b79461e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 06 22:56:55.492940 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:55.492901 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hqqzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 06 22:56:55.493408 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:55.492962 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" podUID="61fd0385-52bf-4df5-afec-e6405b79461e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 06 22:56:56.492380 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:56.492335 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hqqzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 06 22:56:56.492578 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:56.492409 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" podUID="61fd0385-52bf-4df5-afec-e6405b79461e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 06 22:56:57.466518 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:57.466481 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-b5tlr" Jul 06 22:56:57.571031 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:57.570996 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:56:57.571399 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:57.571376 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:56:57.572250 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:56:57.572234 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hqqzn" Jul 06 22:57:50.485169 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.485130 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2"] Jul 06 22:57:50.489022 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.489002 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.492471 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.492436 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bs2s8\"" Jul 06 22:57:50.492471 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.492456 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 22:57:50.492676 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.492495 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 22:57:50.492676 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.492445 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 22:57:50.499095 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.499069 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2"] Jul 06 22:57:50.539358 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.539320 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5accff7a-e5a6-475a-a221-7c85a71855f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.539536 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.539377 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.539536 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.539399 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.539536 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.539451 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw4rf\" (UniqueName: \"kubernetes.io/projected/5accff7a-e5a6-475a-a221-7c85a71855f6-kube-api-access-jw4rf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.539536 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.539496 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.539707 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.539557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640081 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640034 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640081 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640082 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640319 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640116 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4rf\" (UniqueName: \"kubernetes.io/projected/5accff7a-e5a6-475a-a221-7c85a71855f6-kube-api-access-jw4rf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640319 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640168 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640319 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640319 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640288 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5accff7a-e5a6-475a-a221-7c85a71855f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640560 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640535 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640626 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640592 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.640724 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.640656 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.642583 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.642555 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5accff7a-e5a6-475a-a221-7c85a71855f6-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.642996 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.642974 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5accff7a-e5a6-475a-a221-7c85a71855f6-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.647554 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.647522 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4rf\" (UniqueName: \"kubernetes.io/projected/5accff7a-e5a6-475a-a221-7c85a71855f6-kube-api-access-jw4rf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-g6dv2\" (UID: \"5accff7a-e5a6-475a-a221-7c85a71855f6\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.801141 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.801040 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:57:50.948407 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:50.948368 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2"] Jul 06 22:57:50.949872 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:57:50.949839 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5accff7a_e5a6_475a_a221_7c85a71855f6.slice/crio-c4fb755f8791d2b29cbad4d54f5c0057f65d0003d4913ac3c5e965c6ea9d6d36 WatchSource:0}: Error finding container c4fb755f8791d2b29cbad4d54f5c0057f65d0003d4913ac3c5e965c6ea9d6d36: Status 404 returned error can't find the container with id c4fb755f8791d2b29cbad4d54f5c0057f65d0003d4913ac3c5e965c6ea9d6d36 Jul 06 22:57:51.831148 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:51.831048 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" event={"ID":"5accff7a-e5a6-475a-a221-7c85a71855f6","Type":"ContainerStarted","Data":"c4fb755f8791d2b29cbad4d54f5c0057f65d0003d4913ac3c5e965c6ea9d6d36"} Jul 06 22:57:52.840277 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:52.840238 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" event={"ID":"5accff7a-e5a6-475a-a221-7c85a71855f6","Type":"ContainerStarted","Data":"1b8a71d6c10f36e320b7d8156991ed1d454131c9f8c728579029920e6795f156"} Jul 06 22:57:52.857115 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:57:52.857052 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" podStartSLOduration=1.826179402 podStartE2EDuration="2.857034827s" totalImagesPullingTime="1.030855425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:57:52.855846669 +0000 UTC m=+686.155548272" watchObservedRunningTime="2026-07-06 22:57:52.857034827 +0000 UTC m=+686.156736429" Jul 06 22:58:00.801882 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:58:00.801843 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:58:00.802313 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:58:00.801906 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:58:00.814695 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:58:00.814652 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:58:00.893701 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:58:00.893651 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-g6dv2" Jul 06 22:59:21.925591 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:21.925544 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-696b84895b-n86s9"] Jul 06 22:59:21.929137 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:21.929116 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:21.935743 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:21.935716 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-696b84895b-n86s9"] Jul 06 22:59:21.994427 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:21.994381 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kns9\" (UniqueName: \"kubernetes.io/projected/d4d6b552-f397-4a8b-acb4-1edcb6483373-kube-api-access-9kns9\") pod \"authorino-696b84895b-n86s9\" (UID: \"d4d6b552-f397-4a8b-acb4-1edcb6483373\") " pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:21.994611 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:21.994459 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d4d6b552-f397-4a8b-acb4-1edcb6483373-tls-cert\") pod \"authorino-696b84895b-n86s9\" (UID: \"d4d6b552-f397-4a8b-acb4-1edcb6483373\") " pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:22.096003 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:22.095963 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kns9\" (UniqueName: \"kubernetes.io/projected/d4d6b552-f397-4a8b-acb4-1edcb6483373-kube-api-access-9kns9\") pod \"authorino-696b84895b-n86s9\" (UID: \"d4d6b552-f397-4a8b-acb4-1edcb6483373\") " pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:22.096211 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:22.096025 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d4d6b552-f397-4a8b-acb4-1edcb6483373-tls-cert\") pod \"authorino-696b84895b-n86s9\" (UID: \"d4d6b552-f397-4a8b-acb4-1edcb6483373\") " pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:22.098544 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:22.098516 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d4d6b552-f397-4a8b-acb4-1edcb6483373-tls-cert\") pod \"authorino-696b84895b-n86s9\" (UID: \"d4d6b552-f397-4a8b-acb4-1edcb6483373\") " pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:22.104665 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:22.104643 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kns9\" (UniqueName: \"kubernetes.io/projected/d4d6b552-f397-4a8b-acb4-1edcb6483373-kube-api-access-9kns9\") pod \"authorino-696b84895b-n86s9\" (UID: \"d4d6b552-f397-4a8b-acb4-1edcb6483373\") " pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:22.240321 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:22.240229 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-696b84895b-n86s9" Jul 06 22:59:22.394394 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:22.394363 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-696b84895b-n86s9"] Jul 06 22:59:22.395818 ip-10-0-128-38 kubenswrapper[2561]: W0706 22:59:22.395788 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4d6b552_f397_4a8b_acb4_1edcb6483373.slice/crio-02697280ceb9d75cc8a6f0228e8f98fc0dab59967ade3e3b7981ba5407be37e1 WatchSource:0}: Error finding container 02697280ceb9d75cc8a6f0228e8f98fc0dab59967ade3e3b7981ba5407be37e1: Status 404 returned error can't find the container with id 02697280ceb9d75cc8a6f0228e8f98fc0dab59967ade3e3b7981ba5407be37e1 Jul 06 22:59:23.348760 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.348713 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-696b84895b-n86s9" event={"ID":"d4d6b552-f397-4a8b-acb4-1edcb6483373","Type":"ContainerStarted","Data":"5b4438a07c68e66bc41063b600b625d51eff24bb7a6ec6868ccc6eae6709b026"} Jul 06 22:59:23.349241 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.348775 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-696b84895b-n86s9" event={"ID":"d4d6b552-f397-4a8b-acb4-1edcb6483373","Type":"ContainerStarted","Data":"02697280ceb9d75cc8a6f0228e8f98fc0dab59967ade3e3b7981ba5407be37e1"} Jul 06 22:59:23.372564 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.372467 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-696b84895b-n86s9" podStartSLOduration=2.046695897 podStartE2EDuration="2.372443639s" totalImagesPullingTime="325.747742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:59:23.370513527 +0000 UTC m=+776.670215131" watchObservedRunningTime="2026-07-06 22:59:23.372443639 +0000 UTC m=+776.672145245" Jul 06 22:59:23.414227 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.414196 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9d86bdb89-6qmj5"] Jul 06 22:59:23.414451 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.414413 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" podUID="3fd389e3-bac3-4b02-9155-9b5a91d258fd" containerName="authorino" containerID="cri-o://0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2" gracePeriod=30 Jul 06 22:59:23.690153 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.690126 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:59:23.814810 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.814772 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3fd389e3-bac3-4b02-9155-9b5a91d258fd-tls-cert\") pod \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " Jul 06 22:59:23.814992 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.814822 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh6l8\" (UniqueName: \"kubernetes.io/projected/3fd389e3-bac3-4b02-9155-9b5a91d258fd-kube-api-access-kh6l8\") pod \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\" (UID: \"3fd389e3-bac3-4b02-9155-9b5a91d258fd\") " Jul 06 22:59:23.818643 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.817856 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd389e3-bac3-4b02-9155-9b5a91d258fd-kube-api-access-kh6l8" (OuterVolumeSpecName: "kube-api-access-kh6l8") pod "3fd389e3-bac3-4b02-9155-9b5a91d258fd" (UID: "3fd389e3-bac3-4b02-9155-9b5a91d258fd"). InnerVolumeSpecName "kube-api-access-kh6l8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:59:23.827087 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.827049 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fd389e3-bac3-4b02-9155-9b5a91d258fd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "3fd389e3-bac3-4b02-9155-9b5a91d258fd" (UID: "3fd389e3-bac3-4b02-9155-9b5a91d258fd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:59:23.916468 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.916424 2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3fd389e3-bac3-4b02-9155-9b5a91d258fd-tls-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:59:23.916468 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:23.916461 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh6l8\" (UniqueName: \"kubernetes.io/projected/3fd389e3-bac3-4b02-9155-9b5a91d258fd-kube-api-access-kh6l8\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 22:59:24.357334 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.357227 2561 generic.go:356] "Generic (PLEG): container finished" podID="3fd389e3-bac3-4b02-9155-9b5a91d258fd" containerID="0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2" exitCode=0 Jul 06 22:59:24.357334 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.357301 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" Jul 06 22:59:24.357334 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.357317 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" event={"ID":"3fd389e3-bac3-4b02-9155-9b5a91d258fd","Type":"ContainerDied","Data":"0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2"} Jul 06 22:59:24.358022 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.357363 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9d86bdb89-6qmj5" event={"ID":"3fd389e3-bac3-4b02-9155-9b5a91d258fd","Type":"ContainerDied","Data":"5d38de68fb44cb1be54095f666d3f45a42bc65003d3c2e7f422f11491eb614d3"} Jul 06 22:59:24.358022 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.357383 2561 scope.go:117] "RemoveContainer" containerID="0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2" Jul 06 22:59:24.368375 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.368348 2561 scope.go:117] "RemoveContainer" containerID="0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2" Jul 06 22:59:24.368730 ip-10-0-128-38 kubenswrapper[2561]: E0706 22:59:24.368705 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2\": container with ID starting with 0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2 not found: ID does not exist" containerID="0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2" Jul 06 22:59:24.368822 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.368746 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2"} err="failed to get container status \"0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2\": rpc error: code = NotFound desc = could not find container \"0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2\": container with ID starting with 0a71a22663d8432538e12f732a8180548453ed0a04c27b65f6c6889f96510ad2 not found: ID does not exist" Jul 06 22:59:24.385121 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.385084 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9d86bdb89-6qmj5"] Jul 06 22:59:24.390847 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:24.390810 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9d86bdb89-6qmj5"] Jul 06 22:59:25.221140 ip-10-0-128-38 kubenswrapper[2561]: I0706 22:59:25.221109 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fd389e3-bac3-4b02-9155-9b5a91d258fd" path="/var/lib/kubelet/pods/3fd389e3-bac3-4b02-9155-9b5a91d258fd/volumes" Jul 06 23:00:00.146483 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.146447 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp"] Jul 06 23:00:00.147010 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.146824 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fd389e3-bac3-4b02-9155-9b5a91d258fd" containerName="authorino" Jul 06 23:00:00.147010 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.146836 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fd389e3-bac3-4b02-9155-9b5a91d258fd" containerName="authorino" Jul 06 23:00:00.147010 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.146906 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fd389e3-bac3-4b02-9155-9b5a91d258fd" containerName="authorino" Jul 06 23:00:00.149830 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.149811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" Jul 06 23:00:00.152413 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.152384 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 06 23:00:00.152532 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.152446 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 06 23:00:00.153233 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.153221 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhcjn\"" Jul 06 23:00:00.158346 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.158310 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp"] Jul 06 23:00:00.247069 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.247019 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xsz6\" (UniqueName: \"kubernetes.io/projected/87c15747-e6a4-4771-9739-460551d6acd7-kube-api-access-7xsz6\") pod \"maas-api-key-cleanup-29722980-xdzbp\" (UID: \"87c15747-e6a4-4771-9739-460551d6acd7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" Jul 06 23:00:00.347657 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.347624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xsz6\" (UniqueName: \"kubernetes.io/projected/87c15747-e6a4-4771-9739-460551d6acd7-kube-api-access-7xsz6\") pod \"maas-api-key-cleanup-29722980-xdzbp\" (UID: \"87c15747-e6a4-4771-9739-460551d6acd7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" Jul 06 23:00:00.357147 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.357116 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xsz6\" (UniqueName: \"kubernetes.io/projected/87c15747-e6a4-4771-9739-460551d6acd7-kube-api-access-7xsz6\") pod \"maas-api-key-cleanup-29722980-xdzbp\" (UID: \"87c15747-e6a4-4771-9739-460551d6acd7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" Jul 06 23:00:00.461617 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.461578 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" Jul 06 23:00:00.606599 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:00.606571 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp"] Jul 06 23:00:00.608443 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:00:00.608413 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c15747_e6a4_4771_9739_460551d6acd7.slice/crio-ffe404db9a9057fba0a421e3c756bf34902ad91be9c43c1711d2971d22460edb WatchSource:0}: Error finding container ffe404db9a9057fba0a421e3c756bf34902ad91be9c43c1711d2971d22460edb: Status 404 returned error can't find the container with id ffe404db9a9057fba0a421e3c756bf34902ad91be9c43c1711d2971d22460edb Jul 06 23:00:01.574115 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:01.574070 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" event={"ID":"87c15747-e6a4-4771-9739-460551d6acd7","Type":"ContainerStarted","Data":"ffe404db9a9057fba0a421e3c756bf34902ad91be9c43c1711d2971d22460edb"} Jul 06 23:00:26.718799 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:26.718760 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" event={"ID":"87c15747-e6a4-4771-9739-460551d6acd7","Type":"ContainerStarted","Data":"cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600"} Jul 06 23:00:26.735973 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:00:26.735917 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" podStartSLOduration=1.6927545149999998 podStartE2EDuration="26.73590142s" totalImagesPullingTime="25.043146905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:00:26.733756571 +0000 UTC m=+840.033458194" watchObservedRunningTime="2026-07-06 23:00:26.73590142 +0000 UTC m=+840.035603103" Jul 06 23:01:00.010717 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:00.010663 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp"] Jul 06 23:01:00.011279 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:00.010881 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" podUID="87c15747-e6a4-4771-9739-460551d6acd7" containerName="cleanup" containerID="cri-o://cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600" gracePeriod=30 Jul 06 23:01:27.173731 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:27.173627 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:01:27.176221 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:27.176193 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:01:27.179371 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:27.179350 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:01:27.181143 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:27.181121 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:01:30.222256 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:30.222230 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" Jul 06 23:01:30.298960 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:30.298874 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xsz6\" (UniqueName: \"kubernetes.io/projected/87c15747-e6a4-4771-9739-460551d6acd7-kube-api-access-7xsz6\") pod \"87c15747-e6a4-4771-9739-460551d6acd7\" (UID: \"87c15747-e6a4-4771-9739-460551d6acd7\") " Jul 06 23:01:30.301048 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:30.301014 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87c15747-e6a4-4771-9739-460551d6acd7-kube-api-access-7xsz6" (OuterVolumeSpecName: "kube-api-access-7xsz6") pod "87c15747-e6a4-4771-9739-460551d6acd7" (UID: "87c15747-e6a4-4771-9739-460551d6acd7"). InnerVolumeSpecName "kube-api-access-7xsz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:01:30.399899 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:30.399857 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xsz6\" (UniqueName: \"kubernetes.io/projected/87c15747-e6a4-4771-9739-460551d6acd7-kube-api-access-7xsz6\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:01:31.086356 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.086324 2561 generic.go:356] "Generic (PLEG): container finished" podID="87c15747-e6a4-4771-9739-460551d6acd7" containerID="cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600" exitCode=137 Jul 06 23:01:31.086568 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.086366 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" Jul 06 23:01:31.086568 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.086407 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" event={"ID":"87c15747-e6a4-4771-9739-460551d6acd7","Type":"ContainerDied","Data":"cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600"} Jul 06 23:01:31.086568 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.086468 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp" event={"ID":"87c15747-e6a4-4771-9739-460551d6acd7","Type":"ContainerDied","Data":"ffe404db9a9057fba0a421e3c756bf34902ad91be9c43c1711d2971d22460edb"} Jul 06 23:01:31.086568 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.086500 2561 scope.go:117] "RemoveContainer" containerID="cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600" Jul 06 23:01:31.096457 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.096438 2561 scope.go:117] "RemoveContainer" containerID="cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600" Jul 06 23:01:31.096772 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:01:31.096742 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600\": container with ID starting with cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600 not found: ID does not exist" containerID="cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600" Jul 06 23:01:31.096836 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.096785 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600"} err="failed to get container status \"cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600\": rpc error: code = NotFound desc = could not find container \"cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600\": container with ID starting with cf19bdf56057da829142bbc19ce8ee1ff2015e1e6755a80bed587ee0b9112600 not found: ID does not exist" Jul 06 23:01:31.109243 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.109212 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp"] Jul 06 23:01:31.113529 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.113501 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722980-xdzbp"] Jul 06 23:01:31.220812 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:01:31.220774 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87c15747-e6a4-4771-9739-460551d6acd7" path="/var/lib/kubelet/pods/87c15747-e6a4-4771-9739-460551d6acd7/volumes" Jul 06 23:05:17.575296 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:17.575260 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt"] Jul 06 23:05:17.575778 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:17.575488 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" podUID="2eafda25-fd81-4f29-b186-942e07b5f7ee" containerName="manager" containerID="cri-o://d90b52307bcbf5aa820d837c598412ac45f56f90eeba8068b9d8af31fc4bfb6a" gracePeriod=10 Jul 06 23:05:21.394592 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.394500 2561 generic.go:356] "Generic (PLEG): container finished" podID="2eafda25-fd81-4f29-b186-942e07b5f7ee" containerID="d90b52307bcbf5aa820d837c598412ac45f56f90eeba8068b9d8af31fc4bfb6a" exitCode=0 Jul 06 23:05:21.394592 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.394578 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" event={"ID":"2eafda25-fd81-4f29-b186-942e07b5f7ee","Type":"ContainerDied","Data":"d90b52307bcbf5aa820d837c598412ac45f56f90eeba8068b9d8af31fc4bfb6a"} Jul 06 23:05:21.632456 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.632427 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 23:05:21.785655 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.785620 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2eafda25-fd81-4f29-b186-942e07b5f7ee-extensions-socket-volume\") pod \"2eafda25-fd81-4f29-b186-942e07b5f7ee\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " Jul 06 23:05:21.785881 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.785677 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt64n\" (UniqueName: \"kubernetes.io/projected/2eafda25-fd81-4f29-b186-942e07b5f7ee-kube-api-access-wt64n\") pod \"2eafda25-fd81-4f29-b186-942e07b5f7ee\" (UID: \"2eafda25-fd81-4f29-b186-942e07b5f7ee\") " Jul 06 23:05:21.786142 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.786115 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eafda25-fd81-4f29-b186-942e07b5f7ee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2eafda25-fd81-4f29-b186-942e07b5f7ee" (UID: "2eafda25-fd81-4f29-b186-942e07b5f7ee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:05:21.787913 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.787888 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eafda25-fd81-4f29-b186-942e07b5f7ee-kube-api-access-wt64n" (OuterVolumeSpecName: "kube-api-access-wt64n") pod "2eafda25-fd81-4f29-b186-942e07b5f7ee" (UID: "2eafda25-fd81-4f29-b186-942e07b5f7ee"). InnerVolumeSpecName "kube-api-access-wt64n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:05:21.886865 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.886822 2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2eafda25-fd81-4f29-b186-942e07b5f7ee-extensions-socket-volume\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:05:21.886865 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:21.886857 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt64n\" (UniqueName: \"kubernetes.io/projected/2eafda25-fd81-4f29-b186-942e07b5f7ee-kube-api-access-wt64n\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:05:22.401301 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:22.401268 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" Jul 06 23:05:22.401795 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:22.401296 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt" event={"ID":"2eafda25-fd81-4f29-b186-942e07b5f7ee","Type":"ContainerDied","Data":"5033dca1c80eafc6eef62a28211f4026cdec9e1fae75c92a4a4551da8d885e0b"} Jul 06 23:05:22.401795 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:22.401361 2561 scope.go:117] "RemoveContainer" containerID="d90b52307bcbf5aa820d837c598412ac45f56f90eeba8068b9d8af31fc4bfb6a" Jul 06 23:05:22.423900 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:22.423872 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt"] Jul 06 23:05:22.427630 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:22.427604 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9lvtt"] Jul 06 23:05:23.226863 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:05:23.226826 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eafda25-fd81-4f29-b186-942e07b5f7ee" path="/var/lib/kubelet/pods/2eafda25-fd81-4f29-b186-942e07b5f7ee/volumes" Jul 06 23:06:18.681802 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.681764 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp"] Jul 06 23:06:18.682239 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.682128 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eafda25-fd81-4f29-b186-942e07b5f7ee" containerName="manager" Jul 06 23:06:18.682239 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.682139 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eafda25-fd81-4f29-b186-942e07b5f7ee" containerName="manager" Jul 06 23:06:18.682239 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.682147 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87c15747-e6a4-4771-9739-460551d6acd7" containerName="cleanup" Jul 06 23:06:18.682239 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.682153 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="87c15747-e6a4-4771-9739-460551d6acd7" containerName="cleanup" Jul 06 23:06:18.682239 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.682213 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="87c15747-e6a4-4771-9739-460551d6acd7" containerName="cleanup" Jul 06 23:06:18.682239 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.682224 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eafda25-fd81-4f29-b186-942e07b5f7ee" containerName="manager" Jul 06 23:06:18.685247 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.685229 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:18.689107 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.689083 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7md6w\"" Jul 06 23:06:18.696798 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.696774 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp"] Jul 06 23:06:18.698889 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.698862 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dx9d\" (UniqueName: \"kubernetes.io/projected/df9e3161-134d-47d7-a439-ac3bf278703a-kube-api-access-6dx9d\") pod \"kuadrant-operator-controller-manager-86878c769f-m2jjp\" (UID: \"df9e3161-134d-47d7-a439-ac3bf278703a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:18.699031 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.698975 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df9e3161-134d-47d7-a439-ac3bf278703a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m2jjp\" (UID: \"df9e3161-134d-47d7-a439-ac3bf278703a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:18.800129 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.800091 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df9e3161-134d-47d7-a439-ac3bf278703a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m2jjp\" (UID: \"df9e3161-134d-47d7-a439-ac3bf278703a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:18.800319 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.800237 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dx9d\" (UniqueName: \"kubernetes.io/projected/df9e3161-134d-47d7-a439-ac3bf278703a-kube-api-access-6dx9d\") pod \"kuadrant-operator-controller-manager-86878c769f-m2jjp\" (UID: \"df9e3161-134d-47d7-a439-ac3bf278703a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:18.800511 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.800489 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df9e3161-134d-47d7-a439-ac3bf278703a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m2jjp\" (UID: \"df9e3161-134d-47d7-a439-ac3bf278703a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:18.809295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.809263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dx9d\" (UniqueName: \"kubernetes.io/projected/df9e3161-134d-47d7-a439-ac3bf278703a-kube-api-access-6dx9d\") pod \"kuadrant-operator-controller-manager-86878c769f-m2jjp\" (UID: \"df9e3161-134d-47d7-a439-ac3bf278703a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:18.996299 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:18.996206 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:19.142484 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:19.142459 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp"] Jul 06 23:06:19.144561 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:06:19.144524 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9e3161_134d_47d7_a439_ac3bf278703a.slice/crio-1ebf97b77497b653489a7e31db26d14417125a475d843ca80a20e709f940bb39 WatchSource:0}: Error finding container 1ebf97b77497b653489a7e31db26d14417125a475d843ca80a20e709f940bb39: Status 404 returned error can't find the container with id 1ebf97b77497b653489a7e31db26d14417125a475d843ca80a20e709f940bb39 Jul 06 23:06:19.717051 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:19.717011 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" event={"ID":"df9e3161-134d-47d7-a439-ac3bf278703a","Type":"ContainerStarted","Data":"ee655b532728d4ba53488f1f96eeec6cce2ac6394874e7605aadba5803ca4f5b"} Jul 06 23:06:19.717454 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:19.717090 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" event={"ID":"df9e3161-134d-47d7-a439-ac3bf278703a","Type":"ContainerStarted","Data":"1ebf97b77497b653489a7e31db26d14417125a475d843ca80a20e709f940bb39"} Jul 06 23:06:19.717454 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:19.717110 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:06:19.741768 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:19.741699 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" podStartSLOduration=1.741667642 podStartE2EDuration="1.741667642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:06:19.738756197 +0000 UTC m=+1193.038457830" watchObservedRunningTime="2026-07-06 23:06:19.741667642 +0000 UTC m=+1193.041369243" Jul 06 23:06:27.210330 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:27.210300 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:06:27.210859 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:27.210660 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:06:27.215913 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:27.215891 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:06:27.216077 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:27.215946 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:06:30.724838 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:06:30.724806 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m2jjp" Jul 06 23:11:27.249364 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:11:27.249333 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:11:27.249873 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:11:27.249420 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:11:27.254536 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:11:27.254505 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:11:27.254904 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:11:27.254884 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:12:35.509401 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.509359 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92"] Jul 06 23:12:35.512878 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.512858 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.515372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.515347 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-371df82c-openshift-default-dockercfg-wf4qx\"" Jul 06 23:12:35.537091 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.537058 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92"] Jul 06 23:12:35.635741 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.635707 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-podinfo\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.635952 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.635756 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-certs\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.635952 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.635783 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-data\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.635952 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.635865 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-credential-socket\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.635952 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.635935 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-envoy\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.636192 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.635993 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-token\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.636192 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.636036 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d524f2c5-8a52-4702-a2e3-f55416ff9612-istiod-ca-cert\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.636192 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.636076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-socket\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.636192 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.636103 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zngtw\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-kube-api-access-zngtw\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737219 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737181 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-certs\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-data\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737270 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-credential-socket\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737312 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-envoy\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737362 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-token\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737390 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d524f2c5-8a52-4702-a2e3-f55416ff9612-istiod-ca-cert\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737708 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-socket\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737708 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737447 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zngtw\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-kube-api-access-zngtw\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737708 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737507 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-podinfo\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737878 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737733 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-credential-socket\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737878 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737849 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-socket\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.737989 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737950 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-certs\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.738039 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.737998 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-data\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.738124 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.738096 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d524f2c5-8a52-4702-a2e3-f55416ff9612-istiod-ca-cert\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.739848 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.739819 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-envoy\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.740041 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.740020 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-podinfo\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.749314 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.749287 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-token\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.750032 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.750012 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zngtw\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-kube-api-access-zngtw\") pod \"e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.824950 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.824842 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:35.972733 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.972376 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92"] Jul 06 23:12:35.975511 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:12:35.975481 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd524f2c5_8a52_4702_a2e3_f55416ff9612.slice/crio-c71bd5d72e4fb8bbc72c7e814498d07c903bad1dca250fe476234d733090e449 WatchSource:0}: Error finding container c71bd5d72e4fb8bbc72c7e814498d07c903bad1dca250fe476234d733090e449: Status 404 returned error can't find the container with id c71bd5d72e4fb8bbc72c7e814498d07c903bad1dca250fe476234d733090e449 Jul 06 23:12:35.977856 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.977823 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:12:35.977949 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.977893 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:12:35.977949 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:35.977922 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:12:36.869412 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:36.869372 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" event={"ID":"d524f2c5-8a52-4702-a2e3-f55416ff9612","Type":"ContainerStarted","Data":"ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68"} Jul 06 23:12:36.869955 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:36.869455 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" event={"ID":"d524f2c5-8a52-4702-a2e3-f55416ff9612","Type":"ContainerStarted","Data":"c71bd5d72e4fb8bbc72c7e814498d07c903bad1dca250fe476234d733090e449"} Jul 06 23:12:36.891840 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:36.891783 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" podStartSLOduration=1.891764372 podStartE2EDuration="1.891764372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:12:36.889260732 +0000 UTC m=+1570.188962334" watchObservedRunningTime="2026-07-06 23:12:36.891764372 +0000 UTC m=+1570.191465974" Jul 06 23:12:37.826021 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:37.825974 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:37.831379 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:37.831348 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:37.875734 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:37.875675 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:37.876784 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:37.876764 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:41.887527 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:41.887482 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92"] Jul 06 23:12:41.888049 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:41.887729 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" podUID="d524f2c5-8a52-4702-a2e3-f55416ff9612" containerName="istio-proxy" containerID="cri-o://ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68" gracePeriod=30 Jul 06 23:12:47.880933 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.880898 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng"] Jul 06 23:12:47.885161 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.885130 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.892890 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.892854 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c110c670-openshift-default-dockercfg-drdjj\"" Jul 06 23:12:47.904751 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.904720 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng"] Jul 06 23:12:47.955285 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955246 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-credential-socket\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955285 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955285 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be264eb1-1e32-48ec-bd18-48026047d896-istio-podinfo\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955510 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-istio-token\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955510 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955366 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-envoy\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955510 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955393 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be264eb1-1e32-48ec-bd18-48026047d896-istiod-ca-cert\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955510 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955426 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-certs\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955510 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-data\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955510 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955459 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-socket\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:47.955783 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:47.955540 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg466\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-kube-api-access-cg466\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056152 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056109 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg466\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-kube-api-access-cg466\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056186 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-credential-socket\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be264eb1-1e32-48ec-bd18-48026047d896-istio-podinfo\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056228 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-istio-token\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-envoy\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056289 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be264eb1-1e32-48ec-bd18-48026047d896-istiod-ca-cert\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056316 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-certs\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056339 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-data\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056387 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056368 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-socket\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056808 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056645 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-credential-socket\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.056914 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.056890 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-data\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.057038 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.057020 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-certs\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.057233 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.057209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be264eb1-1e32-48ec-bd18-48026047d896-istiod-ca-cert\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.057356 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.057336 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-socket\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.058876 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.058853 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be264eb1-1e32-48ec-bd18-48026047d896-istio-podinfo\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.059132 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.059113 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-envoy\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.064175 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.064150 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-istio-token\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.064325 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.064308 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg466\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-kube-api-access-cg466\") pod \"e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.207643 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.207604 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:48.373894 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.373852 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng"] Jul 06 23:12:48.376142 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:12:48.376112 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe264eb1_1e32_48ec_bd18_48026047d896.slice/crio-e2da5d10485cec41acea1879e734ba8f8a6ddef4589625e0d6590c77d0544e59 WatchSource:0}: Error finding container e2da5d10485cec41acea1879e734ba8f8a6ddef4589625e0d6590c77d0544e59: Status 404 returned error can't find the container with id e2da5d10485cec41acea1879e734ba8f8a6ddef4589625e0d6590c77d0544e59 Jul 06 23:12:48.378337 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.378300 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:12:48.378414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.378389 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:12:48.378454 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.378432 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:12:48.436104 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.436080 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:48.561074 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561036 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-envoy\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561261 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561087 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-data\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561261 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561138 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d524f2c5-8a52-4702-a2e3-f55416ff9612-istiod-ca-cert\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561261 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561171 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-podinfo\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561261 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561206 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-credential-socket\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561502 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561433 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:12:48.561502 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561491 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zngtw\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-kube-api-access-zngtw\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561516 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d524f2c5-8a52-4702-a2e3-f55416ff9612-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:12:48.561606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561531 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-token\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561601 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-certs\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.561789 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561737 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-socket\") pod \"d524f2c5-8a52-4702-a2e3-f55416ff9612\" (UID: \"d524f2c5-8a52-4702-a2e3-f55416ff9612\") " Jul 06 23:12:48.562222 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.561929 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-data" (OuterVolumeSpecName: "istio-data") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:12:48.562222 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.562109 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:12:48.562222 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.562128 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.562222 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.562152 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.562222 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.562169 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d524f2c5-8a52-4702-a2e3-f55416ff9612-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.562568 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.562252 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:12:48.563543 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.563514 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:12:48.563803 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.563783 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:12:48.563922 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.563904 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-token" (OuterVolumeSpecName: "istio-token") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:12:48.564148 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.564134 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-kube-api-access-zngtw" (OuterVolumeSpecName: "kube-api-access-zngtw") pod "d524f2c5-8a52-4702-a2e3-f55416ff9612" (UID: "d524f2c5-8a52-4702-a2e3-f55416ff9612"). InnerVolumeSpecName "kube-api-access-zngtw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:12:48.663581 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.663543 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.663581 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.663576 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zngtw\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-kube-api-access-zngtw\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.663581 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.663586 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.663843 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.663595 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.663843 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.663604 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.663843 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.663611 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d524f2c5-8a52-4702-a2e3-f55416ff9612-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:12:48.944431 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.944393 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" event={"ID":"be264eb1-1e32-48ec-bd18-48026047d896","Type":"ContainerStarted","Data":"4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f"} Jul 06 23:12:48.944959 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.944456 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" event={"ID":"be264eb1-1e32-48ec-bd18-48026047d896","Type":"ContainerStarted","Data":"e2da5d10485cec41acea1879e734ba8f8a6ddef4589625e0d6590c77d0544e59"} Jul 06 23:12:48.945909 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.945879 2561 generic.go:356] "Generic (PLEG): container finished" podID="d524f2c5-8a52-4702-a2e3-f55416ff9612" containerID="ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68" exitCode=0 Jul 06 23:12:48.946016 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.945913 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" Jul 06 23:12:48.946016 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.945962 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" event={"ID":"d524f2c5-8a52-4702-a2e3-f55416ff9612","Type":"ContainerDied","Data":"ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68"} Jul 06 23:12:48.946016 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.946010 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92" event={"ID":"d524f2c5-8a52-4702-a2e3-f55416ff9612","Type":"ContainerDied","Data":"c71bd5d72e4fb8bbc72c7e814498d07c903bad1dca250fe476234d733090e449"} Jul 06 23:12:48.946119 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.946033 2561 scope.go:117] "RemoveContainer" containerID="ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68" Jul 06 23:12:48.956996 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.956974 2561 scope.go:117] "RemoveContainer" containerID="ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68" Jul 06 23:12:48.957281 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:12:48.957262 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68\": container with ID starting with ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68 not found: ID does not exist" containerID="ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68" Jul 06 23:12:48.957331 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.957291 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68"} err="failed to get container status \"ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68\": rpc error: code = NotFound desc = could not find container \"ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68\": container with ID starting with ca83b5744289329d77a0b93ddcbc8da67e2ac727cf035b2bbaf3e9ba0506ec68 not found: ID does not exist" Jul 06 23:12:48.966451 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.966401 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" podStartSLOduration=1.96638605 podStartE2EDuration="1.96638605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:12:48.96436469 +0000 UTC m=+1582.264066321" watchObservedRunningTime="2026-07-06 23:12:48.96638605 +0000 UTC m=+1582.266087651" Jul 06 23:12:48.984563 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.984533 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92"] Jul 06 23:12:48.994073 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:48.994045 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-371df82c-openshift-default-6995d9bbff-tbs92"] Jul 06 23:12:49.208368 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:49.208274 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:49.221808 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:49.221775 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d524f2c5-8a52-4702-a2e3-f55416ff9612" path="/var/lib/kubelet/pods/d524f2c5-8a52-4702-a2e3-f55416ff9612/volumes" Jul 06 23:12:49.222126 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:49.222113 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:49.953078 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:49.953044 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:12:49.954244 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:12:49.954220 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:13:00.013969 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:00.013882 2561 generic.go:356] "Generic (PLEG): container finished" podID="be264eb1-1e32-48ec-bd18-48026047d896" containerID="4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f" exitCode=0 Jul 06 23:13:00.013969 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:00.013954 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" event={"ID":"be264eb1-1e32-48ec-bd18-48026047d896","Type":"ContainerDied","Data":"4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f"} Jul 06 23:13:00.014392 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:00.014317 2561 scope.go:117] "RemoveContainer" containerID="4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f" Jul 06 23:13:00.015320 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:00.015290 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:13:00.015436 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:00.015374 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:13:00.015436 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:00.015417 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:13:00.157425 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:00.157383 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng"] Jul 06 23:13:01.021857 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:01.021807 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" event={"ID":"be264eb1-1e32-48ec-bd18-48026047d896","Type":"ContainerStarted","Data":"f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5"} Jul 06 23:13:01.022327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:01.021853 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" podUID="be264eb1-1e32-48ec-bd18-48026047d896" containerName="istio-proxy" containerID="cri-o://f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5" gracePeriod=30 Jul 06 23:13:04.953494 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:04.953456 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:13:06.206733 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.206677 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4"] Jul 06 23:13:06.207378 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.207350 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d524f2c5-8a52-4702-a2e3-f55416ff9612" containerName="istio-proxy" Jul 06 23:13:06.207378 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.207377 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d524f2c5-8a52-4702-a2e3-f55416ff9612" containerName="istio-proxy" Jul 06 23:13:06.207534 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.207500 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d524f2c5-8a52-4702-a2e3-f55416ff9612" containerName="istio-proxy" Jul 06 23:13:06.210025 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.210000 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.213413 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.213355 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d6f08872-openshift-default-dockercfg-9t52l\"" Jul 06 23:13:06.223942 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.223875 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4"] Jul 06 23:13:06.239912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.239750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-certs\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.239912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.239799 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-envoy\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.239912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.239832 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-socket\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.239912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.239858 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-credential-socket\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.239912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.239889 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-token\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.239912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.239922 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istiod-ca-cert\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.240276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.239968 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-podinfo\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.240276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.240096 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfntv\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-kube-api-access-vfntv\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.240276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.240146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-data\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.299564 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.299543 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:13:06.340866 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.340833 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-istio-token\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341064 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.340874 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be264eb1-1e32-48ec-bd18-48026047d896-istio-podinfo\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341064 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.340901 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-certs\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341064 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.340924 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg466\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-kube-api-access-cg466\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341064 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341040 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-socket\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341290 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341100 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be264eb1-1e32-48ec-bd18-48026047d896-istiod-ca-cert\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341290 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341169 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-envoy\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341290 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341239 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:06.341290 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-credential-socket\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341511 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341303 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-data\") pod \"be264eb1-1e32-48ec-bd18-48026047d896\" (UID: \"be264eb1-1e32-48ec-bd18-48026047d896\") " Jul 06 23:13:06.341511 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341423 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-certs\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.341511 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341460 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-envoy\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.341511 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341491 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-socket\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.341745 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341515 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-credential-socket\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.341745 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341547 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-token\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.341745 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341556 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be264eb1-1e32-48ec-bd18-48026047d896-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:13:06.341745 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istiod-ca-cert\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.341745 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341598 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:06.342001 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.341894 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:06.342287 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.342261 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-credential-socket\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.342385 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.342347 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istiod-ca-cert\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.342543 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.342517 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-socket\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.342922 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.342851 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-podinfo\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.342922 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.342864 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-data" (OuterVolumeSpecName: "istio-data") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:06.342922 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.342858 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-certs\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.343203 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.342943 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfntv\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-kube-api-access-vfntv\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.343203 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343056 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-data\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.343327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343284 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.343327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343301 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.343327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343316 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.343508 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343342 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.343508 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343356 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be264eb1-1e32-48ec-bd18-48026047d896-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.343625 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343594 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-data\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.343781 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343756 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-kube-api-access-cg466" (OuterVolumeSpecName: "kube-api-access-cg466") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "kube-api-access-cg466". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:13:06.344021 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.343999 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-istio-token" (OuterVolumeSpecName: "istio-token") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:13:06.344200 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.344113 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be264eb1-1e32-48ec-bd18-48026047d896-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:13:06.344567 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.344508 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-envoy\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.345426 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.345400 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "be264eb1-1e32-48ec-bd18-48026047d896" (UID: "be264eb1-1e32-48ec-bd18-48026047d896"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:06.346018 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.345992 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-podinfo\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.353940 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.353909 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfntv\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-kube-api-access-vfntv\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.354043 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.353960 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-token\") pod \"e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.444658 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.444622 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be264eb1-1e32-48ec-bd18-48026047d896-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.444658 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.444654 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.444658 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.444664 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be264eb1-1e32-48ec-bd18-48026047d896-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.444912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.444672 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg466\" (UniqueName: \"kubernetes.io/projected/be264eb1-1e32-48ec-bd18-48026047d896-kube-api-access-cg466\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:06.523927 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.523887 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:06.674608 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.674572 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4"] Jul 06 23:13:06.675816 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:13:06.675792 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f321b14_3bbd_4058_b6ed_35d0b59dfcfc.slice/crio-9594d07e9fabaaca2caf7df459a35477c8610f5efc3b83a12e43e8915f69982f WatchSource:0}: Error finding container 9594d07e9fabaaca2caf7df459a35477c8610f5efc3b83a12e43e8915f69982f: Status 404 returned error can't find the container with id 9594d07e9fabaaca2caf7df459a35477c8610f5efc3b83a12e43e8915f69982f Jul 06 23:13:06.677982 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.677944 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:13:06.678080 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.678037 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:13:06.678157 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:06.678087 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:13:07.064756 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.064722 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" event={"ID":"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc","Type":"ContainerStarted","Data":"a0368d4cf355fe70a2357350c1f58a9e46ab73919e931ee6716706f58b5d221f"} Jul 06 23:13:07.064980 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.064784 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" event={"ID":"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc","Type":"ContainerStarted","Data":"9594d07e9fabaaca2caf7df459a35477c8610f5efc3b83a12e43e8915f69982f"} Jul 06 23:13:07.066435 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.066404 2561 generic.go:356] "Generic (PLEG): container finished" podID="be264eb1-1e32-48ec-bd18-48026047d896" containerID="f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5" exitCode=0 Jul 06 23:13:07.066570 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.066446 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" event={"ID":"be264eb1-1e32-48ec-bd18-48026047d896","Type":"ContainerDied","Data":"f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5"} Jul 06 23:13:07.066570 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.066455 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" Jul 06 23:13:07.066570 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.066485 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng" event={"ID":"be264eb1-1e32-48ec-bd18-48026047d896","Type":"ContainerDied","Data":"e2da5d10485cec41acea1879e734ba8f8a6ddef4589625e0d6590c77d0544e59"} Jul 06 23:13:07.066570 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.066515 2561 scope.go:117] "RemoveContainer" containerID="f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5" Jul 06 23:13:07.077044 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.077024 2561 scope.go:117] "RemoveContainer" containerID="4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f" Jul 06 23:13:07.085359 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.085336 2561 scope.go:117] "RemoveContainer" containerID="f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5" Jul 06 23:13:07.085645 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:13:07.085623 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5\": container with ID starting with f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5 not found: ID does not exist" containerID="f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5" Jul 06 23:13:07.085737 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.085659 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5"} err="failed to get container status \"f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5\": rpc error: code = NotFound desc = could not find container \"f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5\": container with ID starting with f567275aded9e8fab1634e2848babb0631db24559a522b9d723ed8d79e9b05f5 not found: ID does not exist" Jul 06 23:13:07.085737 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.085703 2561 scope.go:117] "RemoveContainer" containerID="4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f" Jul 06 23:13:07.086009 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:13:07.085985 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f\": container with ID starting with 4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f not found: ID does not exist" containerID="4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f" Jul 06 23:13:07.086070 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.086021 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f"} err="failed to get container status \"4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f\": rpc error: code = NotFound desc = could not find container \"4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f\": container with ID starting with 4c439a6a450f56dafe54d657d86e5e9991db69274d030267d82db393be703c3f not found: ID does not exist" Jul 06 23:13:07.087434 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.087397 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" podStartSLOduration=1.087385666 podStartE2EDuration="1.087385666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:13:07.086266575 +0000 UTC m=+1600.385968176" watchObservedRunningTime="2026-07-06 23:13:07.087385666 +0000 UTC m=+1600.387087267" Jul 06 23:13:07.102136 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.102104 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng"] Jul 06 23:13:07.106603 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.106579 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c110c670-openshift-default-57c699ccf7-pb2ng"] Jul 06 23:13:07.220652 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.220617 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be264eb1-1e32-48ec-bd18-48026047d896" path="/var/lib/kubelet/pods/be264eb1-1e32-48ec-bd18-48026047d896/volumes" Jul 06 23:13:07.525063 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.525015 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:07.532269 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:07.532236 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:08.073045 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:08.073016 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:08.074062 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:08.074042 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:11.928797 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:11.928757 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4"] Jul 06 23:13:11.929336 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:11.928968 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" podUID="3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" containerName="istio-proxy" containerID="cri-o://a0368d4cf355fe70a2357350c1f58a9e46ab73919e931ee6716706f58b5d221f" gracePeriod=30 Jul 06 23:13:17.128315 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.128271 2561 generic.go:356] "Generic (PLEG): container finished" podID="3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" containerID="a0368d4cf355fe70a2357350c1f58a9e46ab73919e931ee6716706f58b5d221f" exitCode=0 Jul 06 23:13:17.128927 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.128344 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" event={"ID":"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc","Type":"ContainerDied","Data":"a0368d4cf355fe70a2357350c1f58a9e46ab73919e931ee6716706f58b5d221f"} Jul 06 23:13:17.184424 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.184399 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:17.255412 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255325 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istiod-ca-cert\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255412 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255364 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfntv\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-kube-api-access-vfntv\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255412 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255407 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-socket\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255431 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-envoy\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255475 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-token\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255531 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-certs\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255576 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-credential-socket\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255620 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-data\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.255727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255657 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-podinfo\") pod \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\" (UID: \"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc\") " Jul 06 23:13:17.256058 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255729 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:13:17.256058 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255760 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:17.256058 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.255991 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:17.256224 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.256057 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:17.256224 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.256084 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.256224 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.256105 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.256809 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.256787 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:17.257849 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.257823 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:13:17.257958 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.257862 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:13:17.258039 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.257998 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-kube-api-access-vfntv" (OuterVolumeSpecName: "kube-api-access-vfntv") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "kube-api-access-vfntv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:13:17.258098 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.258052 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" (UID: "3f321b14-3bbd-4058-b6ed-35d0b59dfcfc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:13:17.357180 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.357144 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.357180 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.357175 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.357180 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.357186 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.357180 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.357194 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.357459 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.357203 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfntv\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-kube-api-access-vfntv\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.357459 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.357211 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:17.357459 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:17.357221 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:13:18.135465 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:18.135430 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" Jul 06 23:13:18.135979 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:18.135459 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4" event={"ID":"3f321b14-3bbd-4058-b6ed-35d0b59dfcfc","Type":"ContainerDied","Data":"9594d07e9fabaaca2caf7df459a35477c8610f5efc3b83a12e43e8915f69982f"} Jul 06 23:13:18.135979 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:18.135531 2561 scope.go:117] "RemoveContainer" containerID="a0368d4cf355fe70a2357350c1f58a9e46ab73919e931ee6716706f58b5d221f" Jul 06 23:13:18.162594 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:18.162567 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4"] Jul 06 23:13:18.164827 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:18.164804 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d6f08872-openshift-default-7859dc6d7f-f94f4"] Jul 06 23:13:19.221472 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:13:19.221435 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" path="/var/lib/kubelet/pods/3f321b14-3bbd-4058-b6ed-35d0b59dfcfc/volumes" Jul 06 23:14:52.229924 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.229841 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x"] Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230234 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230246 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230259 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be264eb1-1e32-48ec-bd18-48026047d896" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230265 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be264eb1-1e32-48ec-bd18-48026047d896" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230276 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be264eb1-1e32-48ec-bd18-48026047d896" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230282 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be264eb1-1e32-48ec-bd18-48026047d896" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230345 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f321b14-3bbd-4058-b6ed-35d0b59dfcfc" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230353 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="be264eb1-1e32-48ec-bd18-48026047d896" containerName="istio-proxy" Jul 06 23:14:52.230386 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.230359 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="be264eb1-1e32-48ec-bd18-48026047d896" containerName="istio-proxy" Jul 06 23:14:52.233608 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.233586 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.236308 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.236278 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bd8ffd94-openshift-default-dockercfg-cwwwf\"" Jul 06 23:14:52.257829 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.257792 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x"] Jul 06 23:14:52.324869 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.324816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-podinfo\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325065 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.324873 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzpbh\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-kube-api-access-gzpbh\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325065 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.324947 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-socket\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325065 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.324970 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-data\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325249 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.325107 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-credential-socket\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325249 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.325156 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-certs\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325249 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.325195 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istiod-ca-cert\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325249 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.325227 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-envoy\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.325466 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.325257 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-token\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.426642 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-podinfo\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.426642 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426648 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzpbh\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-kube-api-access-gzpbh\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.426886 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426680 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-socket\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.426886 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426750 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-data\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.426886 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426876 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-credential-socket\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427070 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426914 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-certs\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427070 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istiod-ca-cert\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427070 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.426999 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-envoy\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427070 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.427029 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-token\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427281 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.427075 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-socket\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427281 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.427140 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-data\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427413 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.427389 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-credential-socket\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427633 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.427607 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-certs\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.427834 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.427812 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istiod-ca-cert\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.429300 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.429281 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-envoy\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.429666 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.429649 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-podinfo\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.435708 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.435672 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-token\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.435939 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.435923 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzpbh\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-kube-api-access-gzpbh\") pod \"e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.545194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.545095 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:52.697977 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.697947 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x"] Jul 06 23:14:52.700004 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:14:52.699974 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad8dccd_c53f_409d_8463_1e491d7f7cf5.slice/crio-b22760a30e7e124871440a117a8124a771bc0f4ab808f1d12c35280bd8b9f89d WatchSource:0}: Error finding container b22760a30e7e124871440a117a8124a771bc0f4ab808f1d12c35280bd8b9f89d: Status 404 returned error can't find the container with id b22760a30e7e124871440a117a8124a771bc0f4ab808f1d12c35280bd8b9f89d Jul 06 23:14:52.702151 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.702119 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:14:52.702231 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.702187 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:14:52.702231 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:52.702218 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:14:53.672195 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:53.672160 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" event={"ID":"fad8dccd-c53f-409d-8463-1e491d7f7cf5","Type":"ContainerStarted","Data":"efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61"} Jul 06 23:14:53.672702 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:53.672210 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" event={"ID":"fad8dccd-c53f-409d-8463-1e491d7f7cf5","Type":"ContainerStarted","Data":"b22760a30e7e124871440a117a8124a771bc0f4ab808f1d12c35280bd8b9f89d"} Jul 06 23:14:53.691438 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:53.691384 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" podStartSLOduration=1.691368428 podStartE2EDuration="1.691368428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:14:53.68961698 +0000 UTC m=+1706.989318583" watchObservedRunningTime="2026-07-06 23:14:53.691368428 +0000 UTC m=+1706.991070030" Jul 06 23:14:54.546201 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:54.546164 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:54.551490 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:54.551460 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:54.677588 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:54.677552 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:14:54.678452 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:14:54.678429 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:15:00.139334 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.139296 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf"] Jul 06 23:15:00.142886 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.142864 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" Jul 06 23:15:00.145520 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.145488 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 06 23:15:00.145698 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.145668 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 06 23:15:00.146652 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.146634 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhcjn\"" Jul 06 23:15:00.154133 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.154107 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf"] Jul 06 23:15:00.203302 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.203262 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr2q7\" (UniqueName: \"kubernetes.io/projected/1337b0e9-28e5-408c-a56e-418b7f32d857-kube-api-access-cr2q7\") pod \"maas-api-key-cleanup-29722995-hlqwf\" (UID: \"1337b0e9-28e5-408c-a56e-418b7f32d857\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" Jul 06 23:15:00.304819 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.304777 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr2q7\" (UniqueName: \"kubernetes.io/projected/1337b0e9-28e5-408c-a56e-418b7f32d857-kube-api-access-cr2q7\") pod \"maas-api-key-cleanup-29722995-hlqwf\" (UID: \"1337b0e9-28e5-408c-a56e-418b7f32d857\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" Jul 06 23:15:00.313220 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.313184 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr2q7\" (UniqueName: \"kubernetes.io/projected/1337b0e9-28e5-408c-a56e-418b7f32d857-kube-api-access-cr2q7\") pod \"maas-api-key-cleanup-29722995-hlqwf\" (UID: \"1337b0e9-28e5-408c-a56e-418b7f32d857\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" Jul 06 23:15:00.454440 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.454403 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" Jul 06 23:15:00.603043 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.603016 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf"] Jul 06 23:15:00.605109 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:15:00.605078 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1337b0e9_28e5_408c_a56e_418b7f32d857.slice/crio-31bcf3f3456faa05c3d9ec9b6bfaea6d2937275320cdeddd4995783e505fc8fb WatchSource:0}: Error finding container 31bcf3f3456faa05c3d9ec9b6bfaea6d2937275320cdeddd4995783e505fc8fb: Status 404 returned error can't find the container with id 31bcf3f3456faa05c3d9ec9b6bfaea6d2937275320cdeddd4995783e505fc8fb Jul 06 23:15:00.713213 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:00.713065 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" event={"ID":"1337b0e9-28e5-408c-a56e-418b7f32d857","Type":"ContainerStarted","Data":"31bcf3f3456faa05c3d9ec9b6bfaea6d2937275320cdeddd4995783e505fc8fb"} Jul 06 23:15:01.720509 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:01.720465 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" event={"ID":"1337b0e9-28e5-408c-a56e-418b7f32d857","Type":"ContainerStarted","Data":"6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b"} Jul 06 23:15:01.739669 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:01.739601 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" podStartSLOduration=1.7395801400000002 podStartE2EDuration="1.73958014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:15:01.736586857 +0000 UTC m=+1715.036288462" watchObservedRunningTime="2026-07-06 23:15:01.73958014 +0000 UTC m=+1715.039281743" Jul 06 23:15:07.766569 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:07.766483 2561 generic.go:356] "Generic (PLEG): container finished" podID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerID="efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61" exitCode=0 Jul 06 23:15:07.766569 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:07.766553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" event={"ID":"fad8dccd-c53f-409d-8463-1e491d7f7cf5","Type":"ContainerDied","Data":"efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61"} Jul 06 23:15:07.767027 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:07.766934 2561 scope.go:117] "RemoveContainer" containerID="efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61" Jul 06 23:15:07.767934 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:07.767904 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:07.768002 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:07.767977 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:07.768039 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:07.768009 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:08.773954 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:08.773922 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" event={"ID":"fad8dccd-c53f-409d-8463-1e491d7f7cf5","Type":"ContainerStarted","Data":"695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542"} Jul 06 23:15:09.545715 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:09.545647 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:15:09.547276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:09.547234 2561 patch_prober.go:28] interesting pod/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 23:15:09.547429 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:09.547312 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 23:15:09.677462 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:09.677420 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:15:10.545806 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:10.545759 2561 patch_prober.go:28] interesting pod/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 23:15:10.546351 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:10.545830 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 23:15:11.546324 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:11.546272 2561 patch_prober.go:28] interesting pod/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 23:15:11.546897 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:11.546349 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 23:15:12.545968 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:12.545926 2561 patch_prober.go:28] interesting pod/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 23:15:12.546191 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:12.546016 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 23:15:13.550271 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:13.550234 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:15:13.551424 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:13.551399 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:15:14.014596 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:14.014559 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x"] Jul 06 23:15:14.814543 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:14.814499 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" containerID="cri-o://695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542" gracePeriod=30 Jul 06 23:15:20.082120 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.082046 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:15:20.100130 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100099 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-data\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.100275 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100142 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-token\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.100275 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100196 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-podinfo\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.100275 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100218 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzpbh\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-kube-api-access-gzpbh\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.100275 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100269 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-certs\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.101322 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100340 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-socket\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.101322 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100369 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-envoy\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.101322 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100399 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-credential-socket\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.101322 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100427 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istiod-ca-cert\") pod \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\" (UID: \"fad8dccd-c53f-409d-8463-1e491d7f7cf5\") " Jul 06 23:15:20.101322 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.100750 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:15:20.101322 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.101145 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-data" (OuterVolumeSpecName: "istio-data") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:15:20.101717 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.101322 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:15:20.101717 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.101441 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:15:20.102057 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.102020 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:15:20.103173 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.103145 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:15:20.103318 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.103224 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:15:20.103852 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.103819 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-token" (OuterVolumeSpecName: "istio-token") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:15:20.104394 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.104368 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-kube-api-access-gzpbh" (OuterVolumeSpecName: "kube-api-access-gzpbh") pod "fad8dccd-c53f-409d-8463-1e491d7f7cf5" (UID: "fad8dccd-c53f-409d-8463-1e491d7f7cf5"). InnerVolumeSpecName "kube-api-access-gzpbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:15:20.201987 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.201947 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.201987 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.201982 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.201987 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.201991 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.202241 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.202003 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.202241 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.202012 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.202241 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.202021 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.202241 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.202028 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.202241 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.202035 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fad8dccd-c53f-409d-8463-1e491d7f7cf5-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.202241 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.202043 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzpbh\" (UniqueName: \"kubernetes.io/projected/fad8dccd-c53f-409d-8463-1e491d7f7cf5-kube-api-access-gzpbh\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:15:20.852229 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.852193 2561 generic.go:356] "Generic (PLEG): container finished" podID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerID="695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542" exitCode=0 Jul 06 23:15:20.852229 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.852233 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" event={"ID":"fad8dccd-c53f-409d-8463-1e491d7f7cf5","Type":"ContainerDied","Data":"695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542"} Jul 06 23:15:20.852475 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.852242 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" Jul 06 23:15:20.852475 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.852278 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x" event={"ID":"fad8dccd-c53f-409d-8463-1e491d7f7cf5","Type":"ContainerDied","Data":"b22760a30e7e124871440a117a8124a771bc0f4ab808f1d12c35280bd8b9f89d"} Jul 06 23:15:20.852475 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.852300 2561 scope.go:117] "RemoveContainer" containerID="695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542" Jul 06 23:15:20.862171 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.862145 2561 scope.go:117] "RemoveContainer" containerID="efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61" Jul 06 23:15:20.870947 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.870926 2561 scope.go:117] "RemoveContainer" containerID="695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542" Jul 06 23:15:20.871239 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:15:20.871214 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542\": container with ID starting with 695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542 not found: ID does not exist" containerID="695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542" Jul 06 23:15:20.871294 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.871254 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542"} err="failed to get container status \"695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542\": rpc error: code = NotFound desc = could not find container \"695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542\": container with ID starting with 695526693a56dd5fa01ace3c9f929d73ec942e36e9f939d07cc1edcecb15f542 not found: ID does not exist" Jul 06 23:15:20.871294 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.871274 2561 scope.go:117] "RemoveContainer" containerID="efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61" Jul 06 23:15:20.871468 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:15:20.871451 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61\": container with ID starting with efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61 not found: ID does not exist" containerID="efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61" Jul 06 23:15:20.871511 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.871474 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61"} err="failed to get container status \"efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61\": rpc error: code = NotFound desc = could not find container \"efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61\": container with ID starting with efc38685e3859da4bd18c6d6a962e3f571ae0c2f6320c5dacd34638cf93c8e61 not found: ID does not exist" Jul 06 23:15:20.875813 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.875788 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x"] Jul 06 23:15:20.879339 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:20.879317 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bd8ffd94-openshift-default-5495ccc77f-qbc9x"] Jul 06 23:15:21.220171 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:21.220139 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" path="/var/lib/kubelet/pods/fad8dccd-c53f-409d-8463-1e491d7f7cf5/volumes" Jul 06 23:15:33.508732 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.508679 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8"] Jul 06 23:15:33.509112 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.509071 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" Jul 06 23:15:33.509112 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.509082 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" Jul 06 23:15:33.509187 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.509159 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" Jul 06 23:15:33.509187 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.509166 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" Jul 06 23:15:33.511303 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.511285 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.513949 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.513927 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2fa3353e-openshift-default-dockercfg-7q2zt\"" Jul 06 23:15:33.523753 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.523678 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8"] Jul 06 23:15:33.629921 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.629874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-credential-socket\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.629921 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.629922 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-envoy\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.630194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.630062 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-istio-token\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.630194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.630105 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb9jm\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-kube-api-access-sb9jm\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.630194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.630152 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-socket\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.630363 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.630204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-data\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.630363 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.630239 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4702427-4110-4826-9c7c-1a56ca63a074-istiod-ca-cert\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.630363 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.630348 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4702427-4110-4826-9c7c-1a56ca63a074-istio-podinfo\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.630498 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.630376 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-certs\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731025 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.730986 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4702427-4110-4826-9c7c-1a56ca63a074-istio-podinfo\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731025 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731026 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-certs\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731303 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-credential-socket\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731303 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731205 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-envoy\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731303 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-istio-token\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731469 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731319 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb9jm\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-kube-api-access-sb9jm\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731469 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731350 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-socket\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731469 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731409 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-data\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731469 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731441 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4702427-4110-4826-9c7c-1a56ca63a074-istiod-ca-cert\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.731664 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.731541 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-certs\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.732117 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.732059 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-credential-socket\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.732232 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.732168 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-data\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.732232 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.732170 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4702427-4110-4826-9c7c-1a56ca63a074-istiod-ca-cert\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.732327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.732259 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-socket\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.733752 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.733733 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4702427-4110-4826-9c7c-1a56ca63a074-istio-podinfo\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.734105 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.734081 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-envoy\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.751957 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.751925 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-istio-token\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.752110 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.752034 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb9jm\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-kube-api-access-sb9jm\") pod \"e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.822862 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.822757 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:33.973609 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.973579 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8"] Jul 06 23:15:33.975046 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:15:33.975016 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4702427_4110_4826_9c7c_1a56ca63a074.slice/crio-67d69838a1806f48ccca633530d56dc1c844e2816e583ed19b92a261cbe3faef WatchSource:0}: Error finding container 67d69838a1806f48ccca633530d56dc1c844e2816e583ed19b92a261cbe3faef: Status 404 returned error can't find the container with id 67d69838a1806f48ccca633530d56dc1c844e2816e583ed19b92a261cbe3faef Jul 06 23:15:33.977108 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.977070 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:33.977214 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.977176 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:33.977285 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:33.977225 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:34.934318 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:34.934280 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" event={"ID":"e4702427-4110-4826-9c7c-1a56ca63a074","Type":"ContainerStarted","Data":"250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97"} Jul 06 23:15:34.934751 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:34.934345 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" event={"ID":"e4702427-4110-4826-9c7c-1a56ca63a074","Type":"ContainerStarted","Data":"67d69838a1806f48ccca633530d56dc1c844e2816e583ed19b92a261cbe3faef"} Jul 06 23:15:34.954397 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:34.954342 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" podStartSLOduration=1.954325651 podStartE2EDuration="1.954325651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:15:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:15:34.953015123 +0000 UTC m=+1748.252716725" watchObservedRunningTime="2026-07-06 23:15:34.954325651 +0000 UTC m=+1748.254027252" Jul 06 23:15:35.823477 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:35.823436 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:35.829538 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:35.829508 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:35.940065 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:35.940027 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:35.941128 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:35.941109 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:51.484124 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:51.483339 2561 patch_prober.go:28] interesting pod/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:50704->10.133.0.64:15021: read: connection reset by peer" start-of-body= Jul 06 23:15:51.484124 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:51.483411 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:50704->10.133.0.64:15021: read: connection reset by peer" Jul 06 23:15:52.038209 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:52.038178 2561 generic.go:356] "Generic (PLEG): container finished" podID="e4702427-4110-4826-9c7c-1a56ca63a074" containerID="250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97" exitCode=0 Jul 06 23:15:52.038385 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:52.038256 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" event={"ID":"e4702427-4110-4826-9c7c-1a56ca63a074","Type":"ContainerDied","Data":"250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97"} Jul 06 23:15:52.038786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:52.038769 2561 scope.go:117] "RemoveContainer" containerID="250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97" Jul 06 23:15:52.039901 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:52.039869 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:52.039986 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:52.039956 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:52.040044 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:52.039992 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:15:53.049341 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:53.049293 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" event={"ID":"e4702427-4110-4826-9c7c-1a56ca63a074","Type":"ContainerStarted","Data":"618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0"} Jul 06 23:15:53.824114 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:53.824077 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:53.824415 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:53.824123 2561 patch_prober.go:28] interesting pod/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 23:15:53.824599 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:53.824576 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 23:15:54.824058 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:54.824010 2561 patch_prober.go:28] interesting pod/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 23:15:54.824587 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:54.824078 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 23:15:55.824349 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:55.824307 2561 patch_prober.go:28] interesting pod/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 23:15:55.824874 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:55.824380 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 23:15:56.823815 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:56.823767 2561 patch_prober.go:28] interesting pod/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 23:15:56.824037 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:56.823846 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 23:15:57.827368 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:57.827334 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:57.827869 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:57.827679 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:15:57.828551 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:15:57.828526 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:16:00.010562 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:00.010520 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf"] Jul 06 23:16:00.010998 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:00.010738 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" podUID="1337b0e9-28e5-408c-a56e-418b7f32d857" containerName="cleanup" containerID="cri-o://6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b" gracePeriod=30 Jul 06 23:16:27.284230 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:27.284201 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:16:27.286143 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:27.286115 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:16:27.289481 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:27.289452 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:16:27.291276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:27.291257 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:16:30.204251 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.204223 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" Jul 06 23:16:30.269940 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.269847 2561 generic.go:356] "Generic (PLEG): container finished" podID="1337b0e9-28e5-408c-a56e-418b7f32d857" containerID="6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b" exitCode=137 Jul 06 23:16:30.269940 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.269891 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" Jul 06 23:16:30.270129 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.269937 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" event={"ID":"1337b0e9-28e5-408c-a56e-418b7f32d857","Type":"ContainerDied","Data":"6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b"} Jul 06 23:16:30.270129 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.269990 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf" event={"ID":"1337b0e9-28e5-408c-a56e-418b7f32d857","Type":"ContainerDied","Data":"31bcf3f3456faa05c3d9ec9b6bfaea6d2937275320cdeddd4995783e505fc8fb"} Jul 06 23:16:30.270129 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.270017 2561 scope.go:117] "RemoveContainer" containerID="6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b" Jul 06 23:16:30.276727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.276702 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr2q7\" (UniqueName: \"kubernetes.io/projected/1337b0e9-28e5-408c-a56e-418b7f32d857-kube-api-access-cr2q7\") pod \"1337b0e9-28e5-408c-a56e-418b7f32d857\" (UID: \"1337b0e9-28e5-408c-a56e-418b7f32d857\") " Jul 06 23:16:30.278822 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.278791 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1337b0e9-28e5-408c-a56e-418b7f32d857-kube-api-access-cr2q7" (OuterVolumeSpecName: "kube-api-access-cr2q7") pod "1337b0e9-28e5-408c-a56e-418b7f32d857" (UID: "1337b0e9-28e5-408c-a56e-418b7f32d857"). InnerVolumeSpecName "kube-api-access-cr2q7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:16:30.279304 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.279285 2561 scope.go:117] "RemoveContainer" containerID="6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b" Jul 06 23:16:30.279574 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:16:30.279552 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b\": container with ID starting with 6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b not found: ID does not exist" containerID="6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b" Jul 06 23:16:30.279638 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.279587 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b"} err="failed to get container status \"6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b\": rpc error: code = NotFound desc = could not find container \"6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b\": container with ID starting with 6d9c4b4420ea3e8f4a242327763b4fef202689ec3c8676420a8357007c531e4b not found: ID does not exist" Jul 06 23:16:30.377458 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.377421 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr2q7\" (UniqueName: \"kubernetes.io/projected/1337b0e9-28e5-408c-a56e-418b7f32d857-kube-api-access-cr2q7\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:16:30.592345 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.592311 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf"] Jul 06 23:16:30.598174 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:30.598147 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29722995-hlqwf"] Jul 06 23:16:31.220640 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:16:31.220607 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1337b0e9-28e5-408c-a56e-418b7f32d857" path="/var/lib/kubelet/pods/1337b0e9-28e5-408c-a56e-418b7f32d857/volumes" Jul 06 23:17:30.251673 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:30.251636 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8"] Jul 06 23:17:30.252184 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:30.251923 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" containerID="cri-o://618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0" gracePeriod=30 Jul 06 23:17:35.546668 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.546638 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:17:35.593592 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593554 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4702427-4110-4826-9c7c-1a56ca63a074-istiod-ca-cert\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.593798 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593610 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-data\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.593798 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593636 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4702427-4110-4826-9c7c-1a56ca63a074-istio-podinfo\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.593798 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593730 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-envoy\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.593798 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593756 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb9jm\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-kube-api-access-sb9jm\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.593798 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593793 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-credential-socket\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.594078 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593843 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-certs\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.594078 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593888 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-socket\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.594078 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593940 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-istio-token\") pod \"e4702427-4110-4826-9c7c-1a56ca63a074\" (UID: \"e4702427-4110-4826-9c7c-1a56ca63a074\") " Jul 06 23:17:35.594078 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.593946 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4702427-4110-4826-9c7c-1a56ca63a074-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:17:35.594274 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.594099 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:17:35.594327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.594300 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4702427-4110-4826-9c7c-1a56ca63a074-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.594327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.594320 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.594482 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.594456 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:17:35.594554 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.594507 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:17:35.594633 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.594618 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:17:35.596531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.596506 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:17:35.596695 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.596657 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:17:35.596850 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.596830 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4702427-4110-4826-9c7c-1a56ca63a074-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:17:35.596925 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.596830 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-kube-api-access-sb9jm" (OuterVolumeSpecName: "kube-api-access-sb9jm") pod "e4702427-4110-4826-9c7c-1a56ca63a074" (UID: "e4702427-4110-4826-9c7c-1a56ca63a074"). InnerVolumeSpecName "kube-api-access-sb9jm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:17:35.627772 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.627661 2561 generic.go:356] "Generic (PLEG): container finished" podID="e4702427-4110-4826-9c7c-1a56ca63a074" containerID="618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0" exitCode=0 Jul 06 23:17:35.627772 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.627723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" event={"ID":"e4702427-4110-4826-9c7c-1a56ca63a074","Type":"ContainerDied","Data":"618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0"} Jul 06 23:17:35.628018 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.627738 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" Jul 06 23:17:35.628018 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.627834 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8" event={"ID":"e4702427-4110-4826-9c7c-1a56ca63a074","Type":"ContainerDied","Data":"67d69838a1806f48ccca633530d56dc1c844e2816e583ed19b92a261cbe3faef"} Jul 06 23:17:35.628018 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.627867 2561 scope.go:117] "RemoveContainer" containerID="618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0" Jul 06 23:17:35.639065 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.639037 2561 scope.go:117] "RemoveContainer" containerID="250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97" Jul 06 23:17:35.647740 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.647715 2561 scope.go:117] "RemoveContainer" containerID="618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0" Jul 06 23:17:35.648048 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:17:35.648030 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0\": container with ID starting with 618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0 not found: ID does not exist" containerID="618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0" Jul 06 23:17:35.648105 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.648060 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0"} err="failed to get container status \"618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0\": rpc error: code = NotFound desc = could not find container \"618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0\": container with ID starting with 618778ebbb5a1d7d84e1afef5080d0587532f97d2017489931b0117395d805b0 not found: ID does not exist" Jul 06 23:17:35.648105 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.648078 2561 scope.go:117] "RemoveContainer" containerID="250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97" Jul 06 23:17:35.648311 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:17:35.648292 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97\": container with ID starting with 250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97 not found: ID does not exist" containerID="250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97" Jul 06 23:17:35.648374 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.648322 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97"} err="failed to get container status \"250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97\": rpc error: code = NotFound desc = could not find container \"250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97\": container with ID starting with 250621482d64f5fe91e4caa905b084ac5f570e6b72dbb270544e5787e4231a97 not found: ID does not exist" Jul 06 23:17:35.651663 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.651641 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8"] Jul 06 23:17:35.656134 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.656112 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2fa3353e-openshift-default-96df887f-fsxx8"] Jul 06 23:17:35.695448 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.695414 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.695448 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.695447 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.695650 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.695460 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4702427-4110-4826-9c7c-1a56ca63a074-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.695650 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.695471 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.695650 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.695483 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb9jm\" (UniqueName: \"kubernetes.io/projected/e4702427-4110-4826-9c7c-1a56ca63a074-kube-api-access-sb9jm\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.695650 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.695496 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:35.695650 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:35.695506 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4702427-4110-4826-9c7c-1a56ca63a074-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:17:37.220538 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:17:37.220505 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" path="/var/lib/kubelet/pods/e4702427-4110-4826-9c7c-1a56ca63a074/volumes" Jul 06 23:18:24.859636 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.859599 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx"] Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.859985 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.859997 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad8dccd-c53f-409d-8463-1e491d7f7cf5" containerName="istio-proxy" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860009 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860014 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860021 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860030 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860046 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1337b0e9-28e5-408c-a56e-418b7f32d857" containerName="cleanup" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860055 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1337b0e9-28e5-408c-a56e-418b7f32d857" containerName="cleanup" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860132 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" Jul 06 23:18:24.860164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.860143 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1337b0e9-28e5-408c-a56e-418b7f32d857" containerName="cleanup" Jul 06 23:18:24.863313 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.863295 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.865796 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.865773 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2ff615-openshift-default-dockercfg-ws2rf\"" Jul 06 23:18:24.877647 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.877618 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx"] Jul 06 23:18:24.954164 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954122 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-podinfo\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954382 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-certs\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954382 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-token\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954382 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954282 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-socket\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954382 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954343 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-envoy\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954627 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf326d6-9dc0-43b0-b920-ee795252b106-istiod-ca-cert\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954627 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954434 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-credential-socket\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954627 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954462 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-data\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:24.954627 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:24.954500 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggx8z\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-kube-api-access-ggx8z\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.055740 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-envoy\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.055943 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055772 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf326d6-9dc0-43b0-b920-ee795252b106-istiod-ca-cert\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.055943 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055804 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-credential-socket\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.055943 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055823 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-data\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.055943 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx8z\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-kube-api-access-ggx8z\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.055943 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055899 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-podinfo\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056221 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055952 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-certs\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056221 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.055977 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-token\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056221 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.056034 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-socket\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056379 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.056225 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-credential-socket\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056379 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.056263 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-data\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056457 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.056377 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-socket\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056494 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.056462 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-certs\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.056627 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.056608 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf326d6-9dc0-43b0-b920-ee795252b106-istiod-ca-cert\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.058087 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.058063 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-envoy\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.058847 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.058825 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-podinfo\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.063518 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.063494 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-token\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.063637 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.063540 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggx8z\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-kube-api-access-ggx8z\") pod \"e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.174724 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.174662 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:25.331443 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.331416 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx"] Jul 06 23:18:25.334146 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:18:25.334114 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf326d6_9dc0_43b0_b920_ee795252b106.slice/crio-36cfef2afb95d9bfcad4c40e80cc88fe0343d0aebfea00005c7f2ae3aa8a7c39 WatchSource:0}: Error finding container 36cfef2afb95d9bfcad4c40e80cc88fe0343d0aebfea00005c7f2ae3aa8a7c39: Status 404 returned error can't find the container with id 36cfef2afb95d9bfcad4c40e80cc88fe0343d0aebfea00005c7f2ae3aa8a7c39 Jul 06 23:18:25.336824 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.336783 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:25.336894 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.336877 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:25.336933 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.336920 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:25.914220 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.914176 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" event={"ID":"5bf326d6-9dc0-43b0-b920-ee795252b106","Type":"ContainerStarted","Data":"d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd"} Jul 06 23:18:25.914638 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.914246 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" event={"ID":"5bf326d6-9dc0-43b0-b920-ee795252b106","Type":"ContainerStarted","Data":"36cfef2afb95d9bfcad4c40e80cc88fe0343d0aebfea00005c7f2ae3aa8a7c39"} Jul 06 23:18:25.937771 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:25.937681 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" podStartSLOduration=1.937659373 podStartE2EDuration="1.937659373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:18:25.937064692 +0000 UTC m=+1919.236766310" watchObservedRunningTime="2026-07-06 23:18:25.937659373 +0000 UTC m=+1919.237360976" Jul 06 23:18:26.175149 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:26.175112 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:26.179976 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:26.179947 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:26.919850 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:26.919820 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:26.920806 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:26.920786 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:32.921395 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:32.921355 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b"] Jul 06 23:18:32.922031 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:32.921858 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4702427-4110-4826-9c7c-1a56ca63a074" containerName="istio-proxy" Jul 06 23:18:32.925131 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:32.925099 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:32.927709 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:32.927672 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f9d5b5-openshift-default-dockercfg-z6msl\"" Jul 06 23:18:32.936116 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:32.936091 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b"] Jul 06 23:18:33.033616 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.033554 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwq4w\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-kube-api-access-zwq4w\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.033839 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.033624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-credential-socket\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.033839 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.033671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-envoy\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.033972 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.033830 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-token\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.033972 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.033887 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-socket\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.033972 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.033955 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-data\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.034131 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.033985 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be44da5d-7c75-4056-93c9-54ab643c25fc-istiod-ca-cert\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.034131 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.034028 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-podinfo\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.034131 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.034057 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-certs\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135232 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135191 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwq4w\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-kube-api-access-zwq4w\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135232 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-credential-socket\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135255 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-envoy\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135305 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-token\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135331 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-socket\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135386 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-data\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be44da5d-7c75-4056-93c9-54ab643c25fc-istiod-ca-cert\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-podinfo\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-certs\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135919 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135728 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-credential-socket\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135919 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135774 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-socket\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.135919 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.135814 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-certs\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.136082 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.136048 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-data\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.136398 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.136377 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be44da5d-7c75-4056-93c9-54ab643c25fc-istiod-ca-cert\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.138417 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.138388 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-envoy\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.138650 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.138628 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-podinfo\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.147628 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.147600 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-token\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.147814 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.147636 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwq4w\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-kube-api-access-zwq4w\") pod \"e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.238549 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.238451 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:33.414106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.414076 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b"] Jul 06 23:18:33.415950 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:18:33.415908 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe44da5d_7c75_4056_93c9_54ab643c25fc.slice/crio-4e6e0cbd65961dc025ba667225368c6160b32d9c01f5a3f383867e87424593cd WatchSource:0}: Error finding container 4e6e0cbd65961dc025ba667225368c6160b32d9c01f5a3f383867e87424593cd: Status 404 returned error can't find the container with id 4e6e0cbd65961dc025ba667225368c6160b32d9c01f5a3f383867e87424593cd Jul 06 23:18:33.418266 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.418223 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:33.418340 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.418319 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:33.418390 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.418357 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:33.960134 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.960088 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" event={"ID":"be44da5d-7c75-4056-93c9-54ab643c25fc","Type":"ContainerStarted","Data":"3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29"} Jul 06 23:18:33.960629 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.960172 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" event={"ID":"be44da5d-7c75-4056-93c9-54ab643c25fc","Type":"ContainerStarted","Data":"4e6e0cbd65961dc025ba667225368c6160b32d9c01f5a3f383867e87424593cd"} Jul 06 23:18:33.984082 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:33.984022 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" podStartSLOduration=1.9840074909999998 podStartE2EDuration="1.984007491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:18:33.981335149 +0000 UTC m=+1927.281036783" watchObservedRunningTime="2026-07-06 23:18:33.984007491 +0000 UTC m=+1927.283709153" Jul 06 23:18:34.239348 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:34.239259 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:34.244568 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:34.244532 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:34.966072 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:34.966027 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:34.967131 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:34.967108 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:42.961967 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:42.961920 2561 patch_prober.go:28] interesting pod/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 23:18:42.962381 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:42.962001 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 23:18:44.018407 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:44.018371 2561 generic.go:356] "Generic (PLEG): container finished" podID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerID="d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd" exitCode=0 Jul 06 23:18:44.018967 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:44.018445 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" event={"ID":"5bf326d6-9dc0-43b0-b920-ee795252b106","Type":"ContainerDied","Data":"d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd"} Jul 06 23:18:44.019041 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:44.018989 2561 scope.go:117] "RemoveContainer" containerID="d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd" Jul 06 23:18:44.020097 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:44.020070 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:44.020157 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:44.020138 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:44.020196 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:44.020171 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:45.038333 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:45.037294 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" event={"ID":"5bf326d6-9dc0-43b0-b920-ee795252b106","Type":"ContainerStarted","Data":"5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5"} Jul 06 23:18:45.175544 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:45.175507 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:45.175965 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:45.175875 2561 patch_prober.go:28] interesting pod/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 06 23:18:45.176155 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:45.176128 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 06 23:18:46.175263 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:46.175216 2561 patch_prober.go:28] interesting pod/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 06 23:18:46.175944 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:46.175908 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 06 23:18:47.175748 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:47.175674 2561 patch_prober.go:28] interesting pod/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 06 23:18:47.176773 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:47.176352 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 06 23:18:48.175362 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:48.175324 2561 patch_prober.go:28] interesting pod/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 06 23:18:48.175705 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:48.175643 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 06 23:18:49.179412 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:49.179363 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:49.179963 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:49.179942 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:49.180904 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:49.180884 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:18:51.006915 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:51.006864 2561 patch_prober.go:28] interesting pod/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 23:18:51.007455 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:51.006928 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 23:18:52.098157 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:52.098066 2561 generic.go:356] "Generic (PLEG): container finished" podID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerID="3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29" exitCode=0 Jul 06 23:18:52.098559 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:52.098146 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" event={"ID":"be44da5d-7c75-4056-93c9-54ab643c25fc","Type":"ContainerDied","Data":"3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29"} Jul 06 23:18:52.098613 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:52.098597 2561 scope.go:117] "RemoveContainer" containerID="3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29" Jul 06 23:18:52.099863 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:52.099834 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:52.099938 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:52.099921 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:52.099985 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:52.099969 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:18:53.106017 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:53.105979 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" event={"ID":"be44da5d-7c75-4056-93c9-54ab643c25fc","Type":"ContainerStarted","Data":"a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e"} Jul 06 23:18:53.239882 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:53.239826 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:53.240199 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:53.239894 2561 patch_prober.go:28] interesting pod/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 06 23:18:53.240399 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:53.240358 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 06 23:18:54.239863 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:54.239818 2561 patch_prober.go:28] interesting pod/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 06 23:18:54.240503 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:54.240468 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 06 23:18:55.239924 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:55.239872 2561 patch_prober.go:28] interesting pod/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 06 23:18:55.240503 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:55.239949 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 06 23:18:56.239063 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:56.239007 2561 patch_prober.go:28] interesting pod/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 06 23:18:56.239287 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:56.239096 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 06 23:18:57.242602 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:57.242565 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:57.243095 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:57.242919 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:18:57.243584 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:18:57.243567 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:20:27.721780 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:27.721741 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx"] Jul 06 23:20:27.722268 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:27.721986 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" containerID="cri-o://5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5" gracePeriod=30 Jul 06 23:20:32.985197 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:32.985132 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:20:33.111802 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.111762 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-certs\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.111984 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.111820 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf326d6-9dc0-43b0-b920-ee795252b106-istiod-ca-cert\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.111984 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.111846 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-data\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.111984 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.111874 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-token\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.111984 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.111945 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-envoy\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.111984 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.111974 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-credential-socket\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.112235 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112025 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggx8z\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-kube-api-access-ggx8z\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.112235 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112066 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-podinfo\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.112235 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112112 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-socket\") pod \"5bf326d6-9dc0-43b0-b920-ee795252b106\" (UID: \"5bf326d6-9dc0-43b0-b920-ee795252b106\") " Jul 06 23:20:33.112235 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112114 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:33.112446 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112329 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bf326d6-9dc0-43b0-b920-ee795252b106-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:20:33.112503 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112450 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.112503 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112472 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bf326d6-9dc0-43b0-b920-ee795252b106-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.112609 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112541 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:33.112609 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112571 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:33.112735 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.112674 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-data" (OuterVolumeSpecName: "istio-data") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:33.114269 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.114239 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:33.114371 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.114289 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-token" (OuterVolumeSpecName: "istio-token") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:20:33.114534 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.114513 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-kube-api-access-ggx8z" (OuterVolumeSpecName: "kube-api-access-ggx8z") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "kube-api-access-ggx8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:20:33.114612 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.114529 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5bf326d6-9dc0-43b0-b920-ee795252b106" (UID: "5bf326d6-9dc0-43b0-b920-ee795252b106"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:20:33.213342 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.213302 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.213342 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.213337 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.213342 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.213346 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggx8z\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-kube-api-access-ggx8z\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.213592 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.213355 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.213592 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.213364 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.213592 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.213372 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.213592 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.213379 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bf326d6-9dc0-43b0-b920-ee795252b106-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:33.695882 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.695846 2561 generic.go:356] "Generic (PLEG): container finished" podID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerID="5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5" exitCode=0 Jul 06 23:20:33.696102 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.695900 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" Jul 06 23:20:33.696102 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.695926 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" event={"ID":"5bf326d6-9dc0-43b0-b920-ee795252b106","Type":"ContainerDied","Data":"5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5"} Jul 06 23:20:33.696102 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.695979 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx" event={"ID":"5bf326d6-9dc0-43b0-b920-ee795252b106","Type":"ContainerDied","Data":"36cfef2afb95d9bfcad4c40e80cc88fe0343d0aebfea00005c7f2ae3aa8a7c39"} Jul 06 23:20:33.696102 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.696001 2561 scope.go:117] "RemoveContainer" containerID="5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5" Jul 06 23:20:33.704973 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.704952 2561 scope.go:117] "RemoveContainer" containerID="d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd" Jul 06 23:20:33.713087 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.713069 2561 scope.go:117] "RemoveContainer" containerID="5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5" Jul 06 23:20:33.713379 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:20:33.713358 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5\": container with ID starting with 5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5 not found: ID does not exist" containerID="5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5" Jul 06 23:20:33.713440 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.713388 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5"} err="failed to get container status \"5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5\": rpc error: code = NotFound desc = could not find container \"5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5\": container with ID starting with 5d9b80719e9240f736252c2dec345d9a34e5814cd61d30722c996e4589dcaee5 not found: ID does not exist" Jul 06 23:20:33.713440 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.713407 2561 scope.go:117] "RemoveContainer" containerID="d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd" Jul 06 23:20:33.713614 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:20:33.713601 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd\": container with ID starting with d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd not found: ID does not exist" containerID="d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd" Jul 06 23:20:33.713654 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.713619 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd"} err="failed to get container status \"d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd\": rpc error: code = NotFound desc = could not find container \"d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd\": container with ID starting with d79347370e0b4e280a7490de6d2e3938de9658bf5e3139d225e4fae12509c5fd not found: ID does not exist" Jul 06 23:20:33.718708 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.718658 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx"] Jul 06 23:20:33.721304 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:33.721282 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2ff615-openshift-default-b88c65cc-m84wx"] Jul 06 23:20:34.884187 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:34.884147 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b"] Jul 06 23:20:34.884647 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:34.884381 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" containerID="cri-o://a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e" gracePeriod=30 Jul 06 23:20:35.226626 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.226584 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" path="/var/lib/kubelet/pods/5bf326d6-9dc0-43b0-b920-ee795252b106/volumes" Jul 06 23:20:35.674725 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.674669 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv"] Jul 06 23:20:35.675120 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.675106 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" Jul 06 23:20:35.675167 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.675121 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" Jul 06 23:20:35.675167 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.675129 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" Jul 06 23:20:35.675167 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.675135 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" Jul 06 23:20:35.675261 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.675196 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" Jul 06 23:20:35.675261 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.675204 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bf326d6-9dc0-43b0-b920-ee795252b106" containerName="istio-proxy" Jul 06 23:20:35.679772 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.679749 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.682405 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.682383 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cb393d-openshift-default-dockercfg-m5p7l\"" Jul 06 23:20:35.692383 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.692356 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv"] Jul 06 23:20:35.839977 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.839933 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzt98\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-kube-api-access-wzt98\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840153 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.839987 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-certs\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840153 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.840017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-podinfo\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840153 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.840074 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-envoy\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840153 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.840098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-token\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840153 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.840123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2146478-bd6b-4f49-a8dc-80352bf34b84-istiod-ca-cert\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840153 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.840146 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-credential-socket\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840367 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.840208 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-data\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.840367 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.840238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-socket\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.940902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-envoy\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.940949 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-token\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941505 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941083 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2146478-bd6b-4f49-a8dc-80352bf34b84-istiod-ca-cert\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941505 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941146 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-credential-socket\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941505 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-data\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941505 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941244 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-socket\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941505 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzt98\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-kube-api-access-wzt98\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941505 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-certs\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941505 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-podinfo\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941910 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941542 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-credential-socket\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941910 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941643 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-socket\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.941910 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941870 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2146478-bd6b-4f49-a8dc-80352bf34b84-istiod-ca-cert\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.942052 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941946 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-data\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.942052 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.941957 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-certs\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.943344 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.943319 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-envoy\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.943877 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.943858 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-podinfo\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.948905 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.948879 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-token\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.949027 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.948888 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzt98\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-kube-api-access-wzt98\") pod \"e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:35.991278 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:35.991240 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:36.136939 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.136904 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv"] Jul 06 23:20:36.138410 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:20:36.138387 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2146478_bd6b_4f49_a8dc_80352bf34b84.slice/crio-30d585337754b19d920b2102d6d6e22ab83187efc21fc179b1a842aae633daa8 WatchSource:0}: Error finding container 30d585337754b19d920b2102d6d6e22ab83187efc21fc179b1a842aae633daa8: Status 404 returned error can't find the container with id 30d585337754b19d920b2102d6d6e22ab83187efc21fc179b1a842aae633daa8 Jul 06 23:20:36.140728 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.140677 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:36.140807 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.140759 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:36.140807 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.140786 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:36.718020 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.717978 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" event={"ID":"e2146478-bd6b-4f49-a8dc-80352bf34b84","Type":"ContainerStarted","Data":"c7295c314b5b5a185767b8481e5c304b174be913d2a4645187e4471b120f50dc"} Jul 06 23:20:36.718195 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.718042 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" event={"ID":"e2146478-bd6b-4f49-a8dc-80352bf34b84","Type":"ContainerStarted","Data":"30d585337754b19d920b2102d6d6e22ab83187efc21fc179b1a842aae633daa8"} Jul 06 23:20:36.742812 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.742752 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podStartSLOduration=1.742736953 podStartE2EDuration="1.742736953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:20:36.73943342 +0000 UTC m=+2050.039135021" watchObservedRunningTime="2026-07-06 23:20:36.742736953 +0000 UTC m=+2050.042438552" Jul 06 23:20:36.992392 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:36.992307 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:37.997444 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:37.997413 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:38.731076 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:38.731037 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:38.732132 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:38.732112 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:40.151897 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.151831 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:20:40.285124 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285089 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-credential-socket\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285298 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285160 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-certs\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285298 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285186 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-data\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285298 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285219 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-podinfo\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285298 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285247 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-token\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285298 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-envoy\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285536 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285317 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-socket\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285536 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285395 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be44da5d-7c75-4056-93c9-54ab643c25fc-istiod-ca-cert\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285536 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285394 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:40.285536 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285420 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwq4w\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-kube-api-access-zwq4w\") pod \"be44da5d-7c75-4056-93c9-54ab643c25fc\" (UID: \"be44da5d-7c75-4056-93c9-54ab643c25fc\") " Jul 06 23:20:40.285536 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285507 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:40.285853 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285584 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:40.285853 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285732 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be44da5d-7c75-4056-93c9-54ab643c25fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:20:40.285960 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285854 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be44da5d-7c75-4056-93c9-54ab643c25fc-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.285960 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285881 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.285960 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285897 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.285960 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.285911 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.286319 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.286292 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:40.287525 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.287500 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-kube-api-access-zwq4w" (OuterVolumeSpecName: "kube-api-access-zwq4w") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "kube-api-access-zwq4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:20:40.287632 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.287546 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:20:40.287632 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.287564 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:20:40.287737 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.287719 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "be44da5d-7c75-4056-93c9-54ab643c25fc" (UID: "be44da5d-7c75-4056-93c9-54ab643c25fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:20:40.386797 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.386744 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.386797 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.386792 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwq4w\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-kube-api-access-zwq4w\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.386797 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.386804 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.386797 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.386812 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.386797 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.386819 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be44da5d-7c75-4056-93c9-54ab643c25fc-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:20:40.744225 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.744192 2561 generic.go:356] "Generic (PLEG): container finished" podID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerID="a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e" exitCode=0 Jul 06 23:20:40.744448 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.744254 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" Jul 06 23:20:40.744448 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.744286 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" event={"ID":"be44da5d-7c75-4056-93c9-54ab643c25fc","Type":"ContainerDied","Data":"a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e"} Jul 06 23:20:40.744448 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.744342 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b" event={"ID":"be44da5d-7c75-4056-93c9-54ab643c25fc","Type":"ContainerDied","Data":"4e6e0cbd65961dc025ba667225368c6160b32d9c01f5a3f383867e87424593cd"} Jul 06 23:20:40.744448 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.744364 2561 scope.go:117] "RemoveContainer" containerID="a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e" Jul 06 23:20:40.754886 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.754864 2561 scope.go:117] "RemoveContainer" containerID="3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29" Jul 06 23:20:40.763462 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.763444 2561 scope.go:117] "RemoveContainer" containerID="a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e" Jul 06 23:20:40.763765 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:20:40.763743 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e\": container with ID starting with a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e not found: ID does not exist" containerID="a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e" Jul 06 23:20:40.763847 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.763774 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e"} err="failed to get container status \"a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e\": rpc error: code = NotFound desc = could not find container \"a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e\": container with ID starting with a5ebbf359558925be9e6d334a0376c15203ba71d1ad9faafb12dfd92f5907b9e not found: ID does not exist" Jul 06 23:20:40.763847 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.763793 2561 scope.go:117] "RemoveContainer" containerID="3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29" Jul 06 23:20:40.764042 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:20:40.764022 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29\": container with ID starting with 3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29 not found: ID does not exist" containerID="3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29" Jul 06 23:20:40.764085 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.764048 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29"} err="failed to get container status \"3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29\": rpc error: code = NotFound desc = could not find container \"3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29\": container with ID starting with 3881e27532c407fab0e0bda4d68acc2bf333478ae025f90e140128a751b2ff29 not found: ID does not exist" Jul 06 23:20:40.776056 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.773387 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b"] Jul 06 23:20:40.780990 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:40.780957 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f9d5b5-openshift-default-5b64c776b5-dlt8b"] Jul 06 23:20:41.220280 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:41.220240 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" path="/var/lib/kubelet/pods/be44da5d-7c75-4056-93c9-54ab643c25fc/volumes" Jul 06 23:20:43.910837 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.910799 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz"] Jul 06 23:20:43.911414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.911398 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" Jul 06 23:20:43.911474 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.911417 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" Jul 06 23:20:43.911474 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.911432 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" Jul 06 23:20:43.911474 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.911442 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" Jul 06 23:20:43.911569 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.911555 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" Jul 06 23:20:43.911569 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.911567 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="be44da5d-7c75-4056-93c9-54ab643c25fc" containerName="istio-proxy" Jul 06 23:20:43.916558 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.916537 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:43.919480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.919457 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ef756c-openshift-default-dockercfg-8ddfb\"" Jul 06 23:20:43.930402 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:43.930372 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz"] Jul 06 23:20:44.021795 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.021763 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-credential-socket\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.021970 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.021838 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-podinfo\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.021970 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.021871 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpgtc\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-kube-api-access-hpgtc\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.021970 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.021916 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-token\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.022073 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.021994 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-data\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.022073 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.022050 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-socket\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.022147 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.022091 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-envoy\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.022147 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.022125 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-certs\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.022219 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.022169 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istiod-ca-cert\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.123585 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-envoy\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.123765 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123600 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-certs\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.123765 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123635 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istiod-ca-cert\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.123765 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123718 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-credential-socket\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.123909 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-podinfo\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.123909 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpgtc\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-kube-api-access-hpgtc\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124011 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123908 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-token\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124011 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123943 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-data\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124011 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.123983 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-socket\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124160 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.124084 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-certs\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124322 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.124302 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-socket\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124799 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.124771 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istiod-ca-cert\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124897 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.124783 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-credential-socket\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.124897 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.124859 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-data\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.126385 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.126357 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-envoy\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.127128 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.127103 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-podinfo\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.146596 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.146558 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-token\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.147588 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.147563 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpgtc\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-kube-api-access-hpgtc\") pod \"e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.228819 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.228773 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:44.382914 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.382884 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz"] Jul 06 23:20:44.384345 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:20:44.384317 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36590ac4_67ba_4f0d_a7e2_5fbd511cda02.slice/crio-76f3acf4b0aca4e65daca97eb8d269c62a237773d67abc4c587df7126023e367 WatchSource:0}: Error finding container 76f3acf4b0aca4e65daca97eb8d269c62a237773d67abc4c587df7126023e367: Status 404 returned error can't find the container with id 76f3acf4b0aca4e65daca97eb8d269c62a237773d67abc4c587df7126023e367 Jul 06 23:20:44.386438 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.386403 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:44.386507 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.386470 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:44.386507 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.386498 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:44.769441 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.769403 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" event={"ID":"36590ac4-67ba-4f0d-a7e2-5fbd511cda02","Type":"ContainerStarted","Data":"48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd"} Jul 06 23:20:44.769653 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.769463 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" event={"ID":"36590ac4-67ba-4f0d-a7e2-5fbd511cda02","Type":"ContainerStarted","Data":"76f3acf4b0aca4e65daca97eb8d269c62a237773d67abc4c587df7126023e367"} Jul 06 23:20:44.794260 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:44.794194 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" podStartSLOduration=1.794174343 podStartE2EDuration="1.794174343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:20:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:20:44.791136318 +0000 UTC m=+2058.090838002" watchObservedRunningTime="2026-07-06 23:20:44.794174343 +0000 UTC m=+2058.093875943" Jul 06 23:20:45.229216 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:45.229169 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:45.234114 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:45.234084 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:45.775654 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:45.775621 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:45.786815 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:45.786777 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:20:53.849793 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:53.848854 2561 patch_prober.go:28] interesting pod/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": read tcp 10.133.0.2:56788->10.133.0.67:15021: read: connection reset by peer" start-of-body= Jul 06 23:20:53.849793 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:53.848955 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": read tcp 10.133.0.2:56788->10.133.0.67:15021: read: connection reset by peer" Jul 06 23:20:54.838188 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:54.838149 2561 generic.go:356] "Generic (PLEG): container finished" podID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerID="c7295c314b5b5a185767b8481e5c304b174be913d2a4645187e4471b120f50dc" exitCode=0 Jul 06 23:20:54.838435 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:54.838236 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" event={"ID":"e2146478-bd6b-4f49-a8dc-80352bf34b84","Type":"ContainerDied","Data":"c7295c314b5b5a185767b8481e5c304b174be913d2a4645187e4471b120f50dc"} Jul 06 23:20:54.839131 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:54.839108 2561 scope.go:117] "RemoveContainer" containerID="c7295c314b5b5a185767b8481e5c304b174be913d2a4645187e4471b120f50dc" Jul 06 23:20:54.841437 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:54.841400 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:54.841681 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:54.841653 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:54.841931 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:54.841907 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:20:55.851035 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:55.850973 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" event={"ID":"e2146478-bd6b-4f49-a8dc-80352bf34b84","Type":"ContainerStarted","Data":"33ac5ea118769b5718e8d13e469716d1d383c88b55283cc624a9cf6504b730fc"} Jul 06 23:20:55.991788 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:55.991743 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:20:55.992199 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:55.992177 2561 patch_prober.go:28] interesting pod/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 06 23:20:55.992247 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:55.992231 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 06 23:20:56.991999 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:56.991953 2561 patch_prober.go:28] interesting pod/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 06 23:20:56.992414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:56.992025 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 06 23:20:57.991940 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:57.991901 2561 patch_prober.go:28] interesting pod/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 06 23:20:57.992203 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:57.992179 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 06 23:20:58.992528 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:58.992431 2561 patch_prober.go:28] interesting pod/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 06 23:20:58.992528 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:58.992502 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 06 23:20:59.991956 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:59.991910 2561 patch_prober.go:28] interesting pod/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 06 23:20:59.992139 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:20:59.992003 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 06 23:21:00.995815 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:00.995777 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:21:00.996286 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:00.996264 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:21:00.997137 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:00.997115 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:21:01.817843 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:01.817798 2561 patch_prober.go:28] interesting pod/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 23:21:01.818933 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:01.817868 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 23:21:02.899857 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:02.899824 2561 generic.go:356] "Generic (PLEG): container finished" podID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerID="48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd" exitCode=0 Jul 06 23:21:02.900270 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:02.899896 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" event={"ID":"36590ac4-67ba-4f0d-a7e2-5fbd511cda02","Type":"ContainerDied","Data":"48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd"} Jul 06 23:21:02.906874 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:02.901029 2561 scope.go:117] "RemoveContainer" containerID="48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd" Jul 06 23:21:02.907832 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:02.907796 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:02.907995 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:02.907867 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:02.907995 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:02.907899 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:03.907802 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:03.907767 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" event={"ID":"36590ac4-67ba-4f0d-a7e2-5fbd511cda02","Type":"ContainerStarted","Data":"bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16"} Jul 06 23:21:04.229137 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:04.229090 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:21:04.229414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:04.229381 2561 patch_prober.go:28] interesting pod/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 06 23:21:04.229622 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:04.229596 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 06 23:21:05.230214 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:05.230174 2561 patch_prober.go:28] interesting pod/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 06 23:21:05.230738 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:05.230245 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 06 23:21:05.687765 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:05.687726 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv"] Jul 06 23:21:05.688027 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:05.687994 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" containerID="cri-o://33ac5ea118769b5718e8d13e469716d1d383c88b55283cc624a9cf6504b730fc" gracePeriod=30 Jul 06 23:21:06.230285 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:06.230241 2561 patch_prober.go:28] interesting pod/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 06 23:21:06.231340 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:06.230897 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 06 23:21:07.229979 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:07.229936 2561 patch_prober.go:28] interesting pod/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 06 23:21:07.230201 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:07.230011 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 06 23:21:08.233156 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:08.233118 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:21:08.233607 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:08.233510 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:21:08.234167 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:08.234146 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:21:10.953171 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:10.953074 2561 generic.go:356] "Generic (PLEG): container finished" podID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerID="33ac5ea118769b5718e8d13e469716d1d383c88b55283cc624a9cf6504b730fc" exitCode=0 Jul 06 23:21:10.953171 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:10.953136 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" event={"ID":"e2146478-bd6b-4f49-a8dc-80352bf34b84","Type":"ContainerDied","Data":"33ac5ea118769b5718e8d13e469716d1d383c88b55283cc624a9cf6504b730fc"} Jul 06 23:21:10.953651 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:10.953255 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" event={"ID":"e2146478-bd6b-4f49-a8dc-80352bf34b84","Type":"ContainerDied","Data":"30d585337754b19d920b2102d6d6e22ab83187efc21fc179b1a842aae633daa8"} Jul 06 23:21:10.953651 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:10.953283 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30d585337754b19d920b2102d6d6e22ab83187efc21fc179b1a842aae633daa8" Jul 06 23:21:10.953651 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:10.953316 2561 scope.go:117] "RemoveContainer" containerID="c7295c314b5b5a185767b8481e5c304b174be913d2a4645187e4471b120f50dc" Jul 06 23:21:10.960748 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:10.960715 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:21:11.107375 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107337 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-socket\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107582 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107394 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2146478-bd6b-4f49-a8dc-80352bf34b84-istiod-ca-cert\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107582 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107429 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-podinfo\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107582 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107458 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-envoy\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107582 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107491 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-data\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107582 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107570 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-token\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107880 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107603 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-credential-socket\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107880 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107625 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:11.107880 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107638 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzt98\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-kube-api-access-wzt98\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.107880 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107762 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-certs\") pod \"e2146478-bd6b-4f49-a8dc-80352bf34b84\" (UID: \"e2146478-bd6b-4f49-a8dc-80352bf34b84\") " Jul 06 23:21:11.108096 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107903 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2146478-bd6b-4f49-a8dc-80352bf34b84-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:21:11.108096 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.107965 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:11.108217 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.108145 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:11.108217 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.108168 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.108217 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.108189 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.108217 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.108202 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2146478-bd6b-4f49-a8dc-80352bf34b84-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.108517 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.108492 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:11.109729 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.109704 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:11.109862 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.109824 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:21:11.110068 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.110049 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:21:11.110127 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.110092 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-kube-api-access-wzt98" (OuterVolumeSpecName: "kube-api-access-wzt98") pod "e2146478-bd6b-4f49-a8dc-80352bf34b84" (UID: "e2146478-bd6b-4f49-a8dc-80352bf34b84"). InnerVolumeSpecName "kube-api-access-wzt98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:21:11.209106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.209005 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.209106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.209034 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.209106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.209046 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.209106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.209057 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.209106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.209069 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.209106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.209080 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzt98\" (UniqueName: \"kubernetes.io/projected/e2146478-bd6b-4f49-a8dc-80352bf34b84-kube-api-access-wzt98\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:11.959922 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.959888 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv" Jul 06 23:21:11.994251 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.994217 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv"] Jul 06 23:21:12.000008 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:11.999978 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cb393d-openshift-default-795c9c7447-2fdnv"] Jul 06 23:21:12.894670 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:12.894627 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz"] Jul 06 23:21:12.894922 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:12.894871 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" containerID="cri-o://bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16" gracePeriod=30 Jul 06 23:21:13.220815 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.220779 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" path="/var/lib/kubelet/pods/e2146478-bd6b-4f49-a8dc-80352bf34b84/volumes" Jul 06 23:21:13.751788 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.751754 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm"] Jul 06 23:21:13.752189 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.752173 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" Jul 06 23:21:13.752189 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.752190 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" Jul 06 23:21:13.752321 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.752199 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" Jul 06 23:21:13.752321 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.752205 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" Jul 06 23:21:13.752321 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.752298 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" Jul 06 23:21:13.752321 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.752308 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2146478-bd6b-4f49-a8dc-80352bf34b84" containerName="istio-proxy" Jul 06 23:21:13.755346 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.755323 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.760393 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.760373 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2cc96a-openshift-default-dockercfg-bjrh4\"" Jul 06 23:21:13.770116 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.770090 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm"] Jul 06 23:21:13.837348 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837306 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-data\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837348 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istiod-ca-cert\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837616 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837398 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnz5l\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-kube-api-access-vnz5l\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837616 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-credential-socket\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837616 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837473 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-socket\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837616 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837556 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-podinfo\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837876 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837624 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-certs\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837876 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837648 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-token\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.837876 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.837705 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-envoy\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938423 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938378 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-credential-socket\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938623 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-socket\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938623 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938530 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-podinfo\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938623 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938578 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-certs\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938825 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938745 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-token\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938825 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938798 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-envoy\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938944 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-data\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938944 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938885 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istiod-ca-cert\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938944 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938896 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-socket\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.938944 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.938936 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnz5l\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-kube-api-access-vnz5l\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.939191 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.939163 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-certs\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.939376 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.939347 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-data\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.939937 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.939915 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istiod-ca-cert\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.940009 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.939993 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-credential-socket\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.941591 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.941567 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-envoy\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.941846 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.941828 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-podinfo\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.947369 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.947340 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-token\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:13.947884 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:13.947860 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnz5l\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-kube-api-access-vnz5l\") pod \"e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:14.067235 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:14.067141 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:14.214316 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:14.214292 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm"] Jul 06 23:21:14.215860 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:21:14.215828 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa73f8b6_f717_48fe_b456_d95fa3224c7c.slice/crio-7d8497b0f20475b70091e71569e70df37e5e4a49e9fb82f7e6f8b26d2428509c WatchSource:0}: Error finding container 7d8497b0f20475b70091e71569e70df37e5e4a49e9fb82f7e6f8b26d2428509c: Status 404 returned error can't find the container with id 7d8497b0f20475b70091e71569e70df37e5e4a49e9fb82f7e6f8b26d2428509c Jul 06 23:21:14.218012 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:14.217979 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:14.218094 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:14.218054 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:14.218135 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:14.218102 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:14.979357 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:14.979318 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" event={"ID":"fa73f8b6-f717-48fe-b456-d95fa3224c7c","Type":"ContainerStarted","Data":"0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e"} Jul 06 23:21:14.979789 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:14.979382 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" event={"ID":"fa73f8b6-f717-48fe-b456-d95fa3224c7c","Type":"ContainerStarted","Data":"7d8497b0f20475b70091e71569e70df37e5e4a49e9fb82f7e6f8b26d2428509c"} Jul 06 23:21:15.005366 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:15.005309 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" podStartSLOduration=2.00528829 podStartE2EDuration="2.00528829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:21:15.00278503 +0000 UTC m=+2088.302486648" watchObservedRunningTime="2026-07-06 23:21:15.00528829 +0000 UTC m=+2088.304989891" Jul 06 23:21:15.067922 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:15.067879 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:15.073133 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:15.073101 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:15.985754 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:15.985723 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:15.986862 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:15.986839 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:18.147895 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.147864 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:21:18.285914 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.285879 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istiod-ca-cert\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.285914 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.285919 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-socket\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.285989 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-podinfo\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286010 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpgtc\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-kube-api-access-hpgtc\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286062 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-envoy\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286099 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-certs\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286194 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286165 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-data\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286453 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286217 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-credential-socket\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286453 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286240 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-token\") pod \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\" (UID: \"36590ac4-67ba-4f0d-a7e2-5fbd511cda02\") " Jul 06 23:21:18.286453 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286245 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:18.286453 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286422 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:18.286670 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286646 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:18.286747 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286345 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:21:18.286747 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286678 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.286747 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.286721 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.287106 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.287083 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-data" (OuterVolumeSpecName: "istio-data") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:18.288307 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.288283 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:21:18.288491 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.288476 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-token" (OuterVolumeSpecName: "istio-token") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:21:18.288672 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.288646 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:21:18.288794 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.288669 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-kube-api-access-hpgtc" (OuterVolumeSpecName: "kube-api-access-hpgtc") pod "36590ac4-67ba-4f0d-a7e2-5fbd511cda02" (UID: "36590ac4-67ba-4f0d-a7e2-5fbd511cda02"). InnerVolumeSpecName "kube-api-access-hpgtc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:21:18.388044 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.388004 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.388044 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.388041 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.388044 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.388051 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.388284 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.388059 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.388284 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.388067 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpgtc\" (UniqueName: \"kubernetes.io/projected/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-kube-api-access-hpgtc\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.388284 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.388077 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:18.388284 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:18.388084 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36590ac4-67ba-4f0d-a7e2-5fbd511cda02-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:21:19.005607 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.005574 2561 generic.go:356] "Generic (PLEG): container finished" podID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerID="bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16" exitCode=0 Jul 06 23:21:19.005806 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.005618 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" Jul 06 23:21:19.005806 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.005648 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" event={"ID":"36590ac4-67ba-4f0d-a7e2-5fbd511cda02","Type":"ContainerDied","Data":"bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16"} Jul 06 23:21:19.005806 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.005728 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz" event={"ID":"36590ac4-67ba-4f0d-a7e2-5fbd511cda02","Type":"ContainerDied","Data":"76f3acf4b0aca4e65daca97eb8d269c62a237773d67abc4c587df7126023e367"} Jul 06 23:21:19.005806 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.005752 2561 scope.go:117] "RemoveContainer" containerID="bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16" Jul 06 23:21:19.015820 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.015795 2561 scope.go:117] "RemoveContainer" containerID="48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd" Jul 06 23:21:19.032616 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.032585 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz"] Jul 06 23:21:19.032899 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.032826 2561 scope.go:117] "RemoveContainer" containerID="bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16" Jul 06 23:21:19.033228 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:21:19.033205 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16\": container with ID starting with bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16 not found: ID does not exist" containerID="bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16" Jul 06 23:21:19.033328 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.033237 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16"} err="failed to get container status \"bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16\": rpc error: code = NotFound desc = could not find container \"bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16\": container with ID starting with bad1f7e9994d72a9d4691aae2f5ff830c99b2a9dc1e48b6507034dee2bc87a16 not found: ID does not exist" Jul 06 23:21:19.033328 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.033256 2561 scope.go:117] "RemoveContainer" containerID="48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd" Jul 06 23:21:19.033560 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:21:19.033536 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd\": container with ID starting with 48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd not found: ID does not exist" containerID="48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd" Jul 06 23:21:19.033608 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.033568 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd"} err="failed to get container status \"48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd\": rpc error: code = NotFound desc = could not find container \"48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd\": container with ID starting with 48e80aaab636298e75641f3b4d1ff6943dbe5b8dd5e47bf4c8b912d5a063c7fd not found: ID does not exist" Jul 06 23:21:19.035232 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.035212 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ef756c-openshift-default-7fbbb7fffc-g9dmz"] Jul 06 23:21:19.228342 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:19.228303 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" path="/var/lib/kubelet/pods/36590ac4-67ba-4f0d-a7e2-5fbd511cda02/volumes" Jul 06 23:21:21.952314 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.952269 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx"] Jul 06 23:21:21.952858 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.952840 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" Jul 06 23:21:21.952956 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.952863 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" Jul 06 23:21:21.953013 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.953002 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" Jul 06 23:21:21.953064 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.953015 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" Jul 06 23:21:21.956166 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.956142 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:21.959295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.959274 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ca352b-openshift-default-dockercfg-z277j\"" Jul 06 23:21:21.967768 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:21.967744 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx"] Jul 06 23:21:22.126634 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.126580 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-credential-socket\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.126849 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.126760 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-certs\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.126849 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.126802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-data\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.126849 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.126838 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-token\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.126989 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.126917 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-envoy\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.126989 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.126963 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-podinfo\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.127065 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.126996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c258e69-a044-44e0-83f4-cb353e803ebe-istiod-ca-cert\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.127065 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.127036 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-socket\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.127207 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.127181 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv6zc\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-kube-api-access-kv6zc\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.228817 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.228709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-envoy\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.229077 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229056 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-podinfo\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.229193 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229177 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c258e69-a044-44e0-83f4-cb353e803ebe-istiod-ca-cert\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.229306 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229293 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-socket\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.229488 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv6zc\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-kube-api-access-kv6zc\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.229606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229593 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-credential-socket\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.229796 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229776 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-certs\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.229919 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229905 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-data\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.230022 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.230010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-token\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.230474 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.230190 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-credential-socket\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.230474 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.229622 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-socket\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.230474 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.230424 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-data\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.230716 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.230647 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-certs\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.231547 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.231529 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c258e69-a044-44e0-83f4-cb353e803ebe-istiod-ca-cert\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.238038 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.238013 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-envoy\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.238263 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.238237 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-podinfo\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.241452 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.241400 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv6zc\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-kube-api-access-kv6zc\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.241573 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.241514 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-token\") pod \"e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.266923 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.266891 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:22.436606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.436278 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx"] Jul 06 23:21:22.455917 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.455794 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:22.455917 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.455886 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:22.456066 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:22.455934 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:23.038217 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:23.037209 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" event={"ID":"7c258e69-a044-44e0-83f4-cb353e803ebe","Type":"ContainerStarted","Data":"94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776"} Jul 06 23:21:23.038217 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:23.037281 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" event={"ID":"7c258e69-a044-44e0-83f4-cb353e803ebe","Type":"ContainerStarted","Data":"3d5d64186302b185b00de057fe42618001e1233e3fbb89ca031a548395c49404"} Jul 06 23:21:23.070460 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:23.070402 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" podStartSLOduration=2.070385409 podStartE2EDuration="2.070385409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:21:23.068856169 +0000 UTC m=+2096.368557795" watchObservedRunningTime="2026-07-06 23:21:23.070385409 +0000 UTC m=+2096.370087079" Jul 06 23:21:23.267569 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:23.267523 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:23.272867 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:23.272831 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:24.046172 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:24.046131 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:24.051440 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:24.051405 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:27.321174 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:27.321133 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:21:27.324321 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:27.324293 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:21:27.327900 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:27.327874 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:21:27.331525 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:27.331507 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:21:30.085416 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:30.085383 2561 generic.go:356] "Generic (PLEG): container finished" podID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerID="0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e" exitCode=0 Jul 06 23:21:30.085963 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:30.085462 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" event={"ID":"fa73f8b6-f717-48fe-b456-d95fa3224c7c","Type":"ContainerDied","Data":"0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e"} Jul 06 23:21:30.086053 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:30.086036 2561 scope.go:117] "RemoveContainer" containerID="0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e" Jul 06 23:21:30.087110 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:30.087080 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:30.087207 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:30.087148 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:30.087207 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:30.087182 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:30.985666 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:30.985627 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:31.092332 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:31.092292 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" event={"ID":"fa73f8b6-f717-48fe-b456-d95fa3224c7c","Type":"ContainerStarted","Data":"18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada"} Jul 06 23:21:32.068941 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:32.067655 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:32.069331 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:32.069292 2561 patch_prober.go:28] interesting pod/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 06 23:21:32.069492 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:32.069468 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 06 23:21:33.067808 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:33.067749 2561 patch_prober.go:28] interesting pod/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 06 23:21:33.068531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:33.068430 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 06 23:21:34.068596 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:34.068124 2561 patch_prober.go:28] interesting pod/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 06 23:21:34.068596 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:34.068197 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 06 23:21:35.072159 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:35.072117 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:35.072646 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:35.072403 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:35.073179 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:35.073162 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:21:38.138509 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:38.138474 2561 generic.go:356] "Generic (PLEG): container finished" podID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerID="94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776" exitCode=0 Jul 06 23:21:38.138977 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:38.138547 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" event={"ID":"7c258e69-a044-44e0-83f4-cb353e803ebe","Type":"ContainerDied","Data":"94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776"} Jul 06 23:21:38.139087 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:38.139069 2561 scope.go:117] "RemoveContainer" containerID="94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776" Jul 06 23:21:38.140183 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:38.140138 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:38.140300 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:38.140225 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:38.140300 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:38.140280 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:21:39.046875 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:39.046806 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:39.151095 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:39.151048 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" event={"ID":"7c258e69-a044-44e0-83f4-cb353e803ebe","Type":"ContainerStarted","Data":"a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0"} Jul 06 23:21:39.268084 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:39.268033 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:39.269213 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:39.269183 2561 patch_prober.go:28] interesting pod/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 06 23:21:39.269332 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:39.269255 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 06 23:21:40.268026 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:40.267983 2561 patch_prober.go:28] interesting pod/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 06 23:21:40.268479 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:40.268061 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 06 23:21:41.267782 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:41.267750 2561 patch_prober.go:28] interesting pod/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 06 23:21:41.267974 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:41.267825 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 06 23:21:42.267308 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:42.267264 2561 patch_prober.go:28] interesting pod/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 06 23:21:42.267854 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:42.267329 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 06 23:21:43.271907 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:43.271876 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:43.272375 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:43.272212 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:21:43.272913 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:21:43.272894 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:22:01.613606 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:22:01.613561 2561 controller.go:195] "Failed to update lease" err="Put \"https://a627ca4ccd33443d7a2038d2d51ed17f-941f27fd690b701a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-128-38.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 23:22:11.613932 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:22:11.613883 2561 controller.go:195] "Failed to update lease" err="Put \"https://a627ca4ccd33443d7a2038d2d51ed17f-941f27fd690b701a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-128-38.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 23:22:21.615068 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:22:21.614964 2561 controller.go:195] "Failed to update lease" err="Put \"https://a627ca4ccd33443d7a2038d2d51ed17f-941f27fd690b701a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-128-38.ec2.internal?timeout=10s\": context deadline exceeded" Jul 06 23:22:28.309905 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309855 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.309905 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309881 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.309905 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309855 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ftx8p\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.309905 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309903 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.309905 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309879 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309927 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309889 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309870 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309948 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309925 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jmlxb\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309965 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309966 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.309973 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310011 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ca352b-openshift-default-dockercfg-z277j\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310043 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310057 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310084 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310093 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310093 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310125 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310133 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310136 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.310579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310147 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-dockercfg-pc5sr\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310146 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kdnx6\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310179 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310202 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310207 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wkz4c\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310212 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310234 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310253 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310253 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310260 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310280 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310302 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dx78r\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310307 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kjn6w\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310309 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7wkrv\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310327 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cjv6l\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310332 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2cc96a-openshift-default-dockercfg-bjrh4\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310351 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pg49w\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310358 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.311433 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310374 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310374 2561 reflector.go:571] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310394 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7md6w\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310399 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310411 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nwmmk\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310416 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310421 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310437 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310439 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qsc49\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310458 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310461 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kk9pj\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310465 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nrgpx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310463 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"default-dockercfg-bs2s8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310480 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zbk7j\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310487 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310489 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310498 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zg7r9\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310514 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312047 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310519 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310525 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310525 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310552 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310554 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310568 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310587 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"signing-key\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310591 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310595 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310616 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310622 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4v9t\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310626 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310631 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310637 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4hwtm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310649 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310648 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310668 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310671 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xdg5g\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.312606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310679 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310703 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-dockercfg-clp94\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310679 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310729 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qnnlm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310733 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310748 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kth7n\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310734 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310771 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310754 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310755 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmht5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310748 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310762 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310778 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5ggr4\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310808 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310827 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310829 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310842 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2z7cj\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310864 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313169 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310891 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310924 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310926 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310925 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310940 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310939 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6762d\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310952 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310957 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310964 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310973 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310976 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310983 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310984 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.310994 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f7t4d\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311015 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sbtqs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311021 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s9hfh\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311038 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311039 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j9ddx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.313786 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311035 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311057 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311068 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311061 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311080 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qtpz\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311086 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"default-dockercfg-86knk\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311080 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311096 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311037 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311109 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311067 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-njcl6\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311080 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311084 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311122 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311130 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fkc5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311138 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-snqg8\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311150 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311155 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314372 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311165 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t96cf\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311171 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311178 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nxvkb\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311184 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311195 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"service-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:22:28.311190 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T23:22:22Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T23:22:22Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T23:22:22Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T23:22:22Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-128-38.ec2.internal\": Patch \"https://a627ca4ccd33443d7a2038d2d51ed17f-941f27fd690b701a.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-128-38.ec2.internal/status?timeout=10s\": http2: client connection lost" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311209 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311219 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311220 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311229 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311226 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ctk6j\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311245 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311243 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311261 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311252 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.314929 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311264 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7www5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311248 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-console\"/\"console-oauth-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311276 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311288 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311205 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311307 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311309 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311324 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dv6gn\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311327 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311330 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311341 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:28.311357 2561 reflector.go:571] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Jul 06 23:22:28.315406 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:22:28.311403 2561 controller.go:195] "Failed to update lease" err="Put \"https://a627ca4ccd33443d7a2038d2d51ed17f-941f27fd690b701a.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-128-38.ec2.internal?timeout=10s\": http2: client connection lost" Jul 06 23:22:29.124623 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.124589 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 23:22:29.124843 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.124673 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 23:22:29.128154 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.128124 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kk9pj\"" Jul 06 23:22:29.130726 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.130672 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:22:29.133533 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.133510 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 23:22:29.141190 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.141166 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 23:22:29.142182 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.142159 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 23:22:29.149795 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.149770 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nwmmk\"" Jul 06 23:22:29.149965 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.149942 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.158452 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.158425 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 23:22:29.159485 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.159465 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:22:29.159601 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.159469 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 23:22:29.175212 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.175182 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.178109 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.178089 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 23:22:29.178109 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.178107 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sbtqs\"" Jul 06 23:22:29.182660 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.182638 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wkz4c\"" Jul 06 23:22:29.185659 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.185637 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.185787 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.185638 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:22:29.185787 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.185638 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7wkrv\"" Jul 06 23:22:29.187057 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.187040 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.188518 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.188503 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.198284 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.198258 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.213354 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.213328 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 23:22:29.216974 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.216948 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 23:22:29.220949 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.220926 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.223674 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.223653 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.236611 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.236586 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f7t4d\"" Jul 06 23:22:29.237561 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.237544 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-njcl6\"" Jul 06 23:22:29.238352 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.238334 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 23:22:29.240352 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.240334 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nrgpx\"" Jul 06 23:22:29.253221 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.253200 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2z7cj\"" Jul 06 23:22:29.254307 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.254293 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.254622 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.254608 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 23:22:29.262300 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.262278 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 23:22:29.263565 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.263547 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 23:22:29.279543 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.279521 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:22:29.282301 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.282281 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6762d\"" Jul 06 23:22:29.287757 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.287740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 23:22:29.288819 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.288806 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 23:22:29.289073 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.289055 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.289374 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.289358 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.289514 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.289498 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.303563 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.303534 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.311293 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.311268 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.312873 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.312852 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xdg5g\"" Jul 06 23:22:29.316561 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.316541 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.316862 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.316845 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 23:22:29.319831 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.319812 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zbk7j\"" Jul 06 23:22:29.321557 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.321539 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pc5sr\"" Jul 06 23:22:29.327165 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.327144 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 23:22:29.328215 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.328198 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5qtpz\"" Jul 06 23:22:29.333799 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.333780 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 23:22:29.340908 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.340886 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 23:22:29.353772 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.353740 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 23:22:29.357803 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.357782 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 23:22:29.361720 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.361676 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 23:22:29.367537 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.367514 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ca352b-openshift-default-dockercfg-z277j\"" Jul 06 23:22:29.369142 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.369121 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cjv6l\"" Jul 06 23:22:29.372182 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.372165 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s9hfh\"" Jul 06 23:22:29.377147 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.377091 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 23:22:29.377212 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.377197 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 23:22:29.382728 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.382678 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.389306 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.389283 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.394179 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.394153 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:22:29.395637 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.395620 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 23:22:29.409659 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.409631 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 23:22:29.422836 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.422808 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7md6w\"" Jul 06 23:22:29.435941 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.435908 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 23:22:29.443868 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.443843 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 23:22:29.445653 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.445629 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.446631 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.446610 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 23:22:29.460445 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.460410 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jmlxb\"" Jul 06 23:22:29.461263 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.461241 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 23:22:29.467537 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.467508 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dx78r\"" Jul 06 23:22:29.468657 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.468637 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 23:22:29.474183 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.474152 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.474351 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.474253 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 23:22:29.481056 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.481030 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 23:22:29.484147 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.484122 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bs2s8\"" Jul 06 23:22:29.484481 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.484468 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 23:22:29.487159 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.487140 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ctk6j\"" Jul 06 23:22:29.487800 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.487786 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.496842 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.496817 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.508857 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.508832 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.521287 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.521262 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dv6gn\"" Jul 06 23:22:29.529723 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.529660 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.534757 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.534731 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.536440 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.536416 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.545578 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.545548 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 23:22:29.558842 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.558811 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j9ddx\"" Jul 06 23:22:29.564702 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.564666 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.566807 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.566788 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5ggr4\"" Jul 06 23:22:29.568549 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.568529 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ftx8p\"" Jul 06 23:22:29.569599 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.569584 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4v9t\"" Jul 06 23:22:29.578155 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.578126 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-snqg8\"" Jul 06 23:22:29.581962 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.581934 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nxvkb\"" Jul 06 23:22:29.582120 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.582024 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vmht5\"" Jul 06 23:22:29.593041 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.593010 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.604506 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.604476 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.604822 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.604804 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 23:22:29.605073 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.605056 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:22:29.606563 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.606544 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 23:22:29.609402 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.609388 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 23:22:29.610872 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.610856 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.615307 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.615287 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.623885 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.623860 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-86knk\"" Jul 06 23:22:29.625461 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.625442 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qnnlm\"" Jul 06 23:22:29.630909 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.630843 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.640779 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.640751 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4hwtm\"" Jul 06 23:22:29.650099 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.650069 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 23:22:29.650274 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.650126 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zg7r9\"" Jul 06 23:22:29.663014 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.662984 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 23:22:29.666344 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.666319 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 23:22:29.680914 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.680881 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.685627 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.685602 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kth7n\"" Jul 06 23:22:29.690255 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.690230 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qsc49\"" Jul 06 23:22:29.693549 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.693528 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 23:22:29.706526 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.706500 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.719035 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.718998 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 23:22:29.719718 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.719699 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 23:22:29.719905 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.719892 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 23:22:29.724728 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.724707 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kjn6w\"" Jul 06 23:22:29.726352 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.726337 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.744785 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.744749 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7www5\"" Jul 06 23:22:29.753833 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.753805 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 23:22:29.765933 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.765906 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.766761 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.766737 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 23:22:29.772832 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.772809 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.786127 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.786100 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 23:22:29.790962 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.790943 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 23:22:29.792166 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.792150 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-clp94\"" Jul 06 23:22:29.803414 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.803383 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 23:22:29.808393 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.808365 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pg49w\"" Jul 06 23:22:29.812102 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.812077 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.825532 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.825498 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 23:22:29.852234 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.852199 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 23:22:29.865907 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.865875 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.884949 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.884859 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:22:29.905715 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.905655 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.925900 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.925867 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 23:22:29.945253 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.945219 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 23:22:29.965054 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.965023 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2cc96a-openshift-default-dockercfg-bjrh4\"" Jul 06 23:22:29.994606 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:29.994567 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 23:22:30.005191 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.005160 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kdnx6\"" Jul 06 23:22:30.025565 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.025533 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 23:22:30.045808 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.045778 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 23:22:30.065218 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.065190 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 23:22:30.085104 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.085074 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 23:22:30.105101 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.105063 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fkc5\"" Jul 06 23:22:30.125778 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.125746 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 23:22:30.145443 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.145374 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:22:30.165702 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.165663 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:22:30.184717 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.184658 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:22:30.205845 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.205810 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 23:22:30.224959 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.224931 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 23:22:30.245063 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:22:30.245036 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t96cf\"" Jul 06 23:26:27.358714 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:26:27.358672 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:26:27.362818 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:26:27.362795 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:26:27.363483 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:26:27.363461 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:26:27.367764 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:26:27.367745 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-sprk8_720f5ab6-a3bb-4e67-8125-7d222c1d9b9d/ovn-acl-logging/0.log" Jul 06 23:27:27.510579 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:27.510533 2561 scope.go:117] "RemoveContainer" containerID="33ac5ea118769b5718e8d13e469716d1d383c88b55283cc624a9cf6504b730fc" Jul 06 23:27:42.334979 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:42.334932 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm"] Jul 06 23:27:42.335448 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:42.335293 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" containerID="cri-o://18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada" gracePeriod=30 Jul 06 23:27:47.608679 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.608609 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:27:47.680213 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680174 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-credential-socket\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680213 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680213 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-certs\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680239 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istiod-ca-cert\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680257 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-envoy\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680324 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-token\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680387 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-podinfo\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680426 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-data\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680452 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnz5l\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-kube-api-access-vnz5l\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680480 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680454 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:47.680878 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680506 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-socket\") pod \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\" (UID: \"fa73f8b6-f717-48fe-b456-d95fa3224c7c\") " Jul 06 23:27:47.680878 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680586 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:47.680878 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680724 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:27:47.681031 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.680946 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:47.681092 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.681042 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.681092 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.681065 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.681092 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.681082 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.681212 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.681095 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.681800 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.681770 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:47.682732 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.682702 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:27:47.682872 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.682831 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-kube-api-access-vnz5l" (OuterVolumeSpecName: "kube-api-access-vnz5l") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "kube-api-access-vnz5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:27:47.683118 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.683082 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:27:47.683197 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.683127 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa73f8b6-f717-48fe-b456-d95fa3224c7c" (UID: "fa73f8b6-f717-48fe-b456-d95fa3224c7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:47.782005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.781964 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.782005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.781997 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.782005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.782007 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.782005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.782018 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa73f8b6-f717-48fe-b456-d95fa3224c7c-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:47.782290 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:47.782026 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnz5l\" (UniqueName: \"kubernetes.io/projected/fa73f8b6-f717-48fe-b456-d95fa3224c7c-kube-api-access-vnz5l\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:48.290821 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.290784 2561 generic.go:356] "Generic (PLEG): container finished" podID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerID="18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada" exitCode=0 Jul 06 23:27:48.291005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.290846 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" Jul 06 23:27:48.291005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.290866 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" event={"ID":"fa73f8b6-f717-48fe-b456-d95fa3224c7c","Type":"ContainerDied","Data":"18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada"} Jul 06 23:27:48.291005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.290921 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm" event={"ID":"fa73f8b6-f717-48fe-b456-d95fa3224c7c","Type":"ContainerDied","Data":"7d8497b0f20475b70091e71569e70df37e5e4a49e9fb82f7e6f8b26d2428509c"} Jul 06 23:27:48.291005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.290943 2561 scope.go:117] "RemoveContainer" containerID="18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada" Jul 06 23:27:48.303656 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.303627 2561 scope.go:117] "RemoveContainer" containerID="0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e" Jul 06 23:27:48.312499 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.312477 2561 scope.go:117] "RemoveContainer" containerID="18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada" Jul 06 23:27:48.312803 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:27:48.312785 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada\": container with ID starting with 18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada not found: ID does not exist" containerID="18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada" Jul 06 23:27:48.312858 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.312813 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada"} err="failed to get container status \"18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada\": rpc error: code = NotFound desc = could not find container \"18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada\": container with ID starting with 18c8681d24e46a8eb68fa1d983b96d5d53f46e7e7f493da63a576c0771359ada not found: ID does not exist" Jul 06 23:27:48.312858 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.312844 2561 scope.go:117] "RemoveContainer" containerID="0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e" Jul 06 23:27:48.313070 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:27:48.313055 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e\": container with ID starting with 0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e not found: ID does not exist" containerID="0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e" Jul 06 23:27:48.313118 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.313073 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e"} err="failed to get container status \"0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e\": rpc error: code = NotFound desc = could not find container \"0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e\": container with ID starting with 0d3f0867effef67169ee28c86921c81c241a92d04e577d903ff8c9745bb46e6e not found: ID does not exist" Jul 06 23:27:48.317284 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.317256 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm"] Jul 06 23:27:48.323338 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:48.323310 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-2cc96a-openshift-default-59dc5ddf4c-72qbm"] Jul 06 23:27:49.012665 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:49.012631 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx"] Jul 06 23:27:49.013133 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:49.012881 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" containerID="cri-o://a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0" gracePeriod=30 Jul 06 23:27:49.221155 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:49.221119 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" path="/var/lib/kubelet/pods/fa73f8b6-f717-48fe-b456-d95fa3224c7c/volumes" Jul 06 23:27:49.680849 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:49.680812 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-696b84895b-n86s9_d4d6b552-f397-4a8b-acb4-1edcb6483373/authorino/0.log" Jul 06 23:27:54.239838 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.239800 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hpp57_2ebebcc1-626f-4f0d-a317-b1c8edbb4349/manager/0.log" Jul 06 23:27:54.276516 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.276446 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:27:54.327720 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.327646 2561 generic.go:356] "Generic (PLEG): container finished" podID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerID="a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0" exitCode=0 Jul 06 23:27:54.327720 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.327720 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" Jul 06 23:27:54.327961 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.327720 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" event={"ID":"7c258e69-a044-44e0-83f4-cb353e803ebe","Type":"ContainerDied","Data":"a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0"} Jul 06 23:27:54.327961 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.327816 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx" event={"ID":"7c258e69-a044-44e0-83f4-cb353e803ebe","Type":"ContainerDied","Data":"3d5d64186302b185b00de057fe42618001e1233e3fbb89ca031a548395c49404"} Jul 06 23:27:54.327961 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.327849 2561 scope.go:117] "RemoveContainer" containerID="a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0" Jul 06 23:27:54.337271 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.337249 2561 scope.go:117] "RemoveContainer" containerID="94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776" Jul 06 23:27:54.345797 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.345774 2561 scope.go:117] "RemoveContainer" containerID="a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0" Jul 06 23:27:54.346078 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:27:54.346054 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0\": container with ID starting with a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0 not found: ID does not exist" containerID="a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0" Jul 06 23:27:54.346160 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.346095 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0"} err="failed to get container status \"a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0\": rpc error: code = NotFound desc = could not find container \"a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0\": container with ID starting with a93c9e2cf0543e87c75213eb33f221bcfebccb628945a0c61cbfdb7b69cce2e0 not found: ID does not exist" Jul 06 23:27:54.346160 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.346122 2561 scope.go:117] "RemoveContainer" containerID="94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776" Jul 06 23:27:54.346362 ip-10-0-128-38 kubenswrapper[2561]: E0706 23:27:54.346346 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776\": container with ID starting with 94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776 not found: ID does not exist" containerID="94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776" Jul 06 23:27:54.346410 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.346369 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776"} err="failed to get container status \"94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776\": rpc error: code = NotFound desc = could not find container \"94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776\": container with ID starting with 94ed6668d7e7932d2b1e72268ace7880cc57a008c6f4a1ff84089aad61461776 not found: ID does not exist" Jul 06 23:27:54.446806 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.446754 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-data\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447026 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.446833 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv6zc\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-kube-api-access-kv6zc\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447026 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.446871 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-token\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447026 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.446903 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-certs\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447065 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-socket\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447115 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c258e69-a044-44e0-83f4-cb353e803ebe-istiod-ca-cert\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447146 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-envoy\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447180 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-credential-socket\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447276 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447240 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-podinfo\") pod \"7c258e69-a044-44e0-83f4-cb353e803ebe\" (UID: \"7c258e69-a044-44e0-83f4-cb353e803ebe\") " Jul 06 23:27:54.447557 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447306 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:54.447557 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447480 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:54.447557 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447521 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:54.447719 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447572 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c258e69-a044-44e0-83f4-cb353e803ebe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:27:54.447719 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447652 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-certs\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.447719 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447670 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-workload-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.447719 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447704 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-credential-socket\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.447950 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.447930 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:54.449275 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.449246 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-kube-api-access-kv6zc" (OuterVolumeSpecName: "kube-api-access-kv6zc") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "kube-api-access-kv6zc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:27:54.449415 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.449310 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:27:54.449572 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.449552 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:27:54.449738 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.449710 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c258e69-a044-44e0-83f4-cb353e803ebe" (UID: "7c258e69-a044-44e0-83f4-cb353e803ebe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:27:54.481423 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.481394 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cbrc5_a1e21ab9-d7e9-4f0d-9a03-8699c11061b6/server/0.log" Jul 06 23:27:54.549000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.548914 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-podinfo\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.549000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.548942 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-data\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.549000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.548951 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv6zc\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-kube-api-access-kv6zc\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.549000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.548963 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.549000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.548973 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c258e69-a044-44e0-83f4-cb353e803ebe-istiod-ca-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.549000 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.548981 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c258e69-a044-44e0-83f4-cb353e803ebe-istio-envoy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 06 23:27:54.593626 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.593567 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wvqp8_650d898d-1738-4b98-b1df-7ce751e04ab8/manager/0.log" Jul 06 23:27:54.652030 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.651994 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx"] Jul 06 23:27:54.657896 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.657854 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ca352b-openshift-default-655bd94555-wf8nx"] Jul 06 23:27:54.705469 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.705444 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-cs6d2_6c93e238-ab31-4458-8a69-b797e29aa41b/copy-manifests/0.log" Jul 06 23:27:54.712077 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:54.712049 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-cs6d2_6c93e238-ab31-4458-8a69-b797e29aa41b/manager/0.log" Jul 06 23:27:55.220352 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.220318 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" path="/var/lib/kubelet/pods/7c258e69-a044-44e0-83f4-cb353e803ebe/volumes" Jul 06 23:27:55.696958 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.696927 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx_d07fcd1e-5d05-4cec-a74c-5cf8464a0499/util/0.log" Jul 06 23:27:55.703461 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.703434 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx_d07fcd1e-5d05-4cec-a74c-5cf8464a0499/pull/0.log" Jul 06 23:27:55.709514 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.709483 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx_d07fcd1e-5d05-4cec-a74c-5cf8464a0499/extract/0.log" Jul 06 23:27:55.822093 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.822064 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj_6527ff3e-840b-47d3-9500-86dc480b51c3/util/0.log" Jul 06 23:27:55.828819 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.828793 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj_6527ff3e-840b-47d3-9500-86dc480b51c3/pull/0.log" Jul 06 23:27:55.834990 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.834958 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj_6527ff3e-840b-47d3-9500-86dc480b51c3/extract/0.log" Jul 06 23:27:55.943938 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.943901 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2_7e30516f-e531-4072-a38b-67fe2f38ff4e/util/0.log" Jul 06 23:27:55.949601 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.949532 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2_7e30516f-e531-4072-a38b-67fe2f38ff4e/pull/0.log" Jul 06 23:27:55.955471 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:55.955454 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2_7e30516f-e531-4072-a38b-67fe2f38ff4e/extract/0.log" Jul 06 23:27:56.066649 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.066616 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb_7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187/util/0.log" Jul 06 23:27:56.073271 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.073243 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb_7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187/pull/0.log" Jul 06 23:27:56.079916 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.079890 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb_7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187/extract/0.log" Jul 06 23:27:56.193468 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.193436 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-696b84895b-n86s9_d4d6b552-f397-4a8b-acb4-1edcb6483373/authorino/0.log" Jul 06 23:27:56.304260 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.304164 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qhhjt_3bfc6fc4-7482-46af-bf1d-cead4d5104b9/manager/0.log" Jul 06 23:27:56.516610 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.516586 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6n45n_0910a5f2-2bbb-49d8-8c37-ff472d59504e/kuadrant-console-plugin/0.log" Jul 06 23:27:56.633405 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.633321 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p74df_c9006499-1789-4d73-99dd-0f422c4c8a3f/registry-server/0.log" Jul 06 23:27:56.745278 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.745246 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-m2jjp_df9e3161-134d-47d7-a439-ac3bf278703a/manager/0.log" Jul 06 23:27:56.975243 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:56.975215 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-g4lpc_55e16c41-d39f-454b-8307-624b4925bdbe/manager/0.log" Jul 06 23:27:57.316875 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:57.316775 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5bp7_cdeabf53-1e52-4feb-9cb4-dce852a82f64/istio-proxy/0.log" Jul 06 23:27:57.652327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:57.652246 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54f95589f8-jqc4s_e20fa170-f482-45a3-ac02-391f1a736daf/kube-auth-proxy/0.log" Jul 06 23:27:57.772456 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:57.772425 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hqqzn_61fd0385-52bf-4df5-afec-e6405b79461e/istio-proxy/1.log" Jul 06 23:27:58.002803 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:58.002769 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-b5tlr_3d3caabc-69e1-4913-94b9-c9ca44a92f3a/payload-processing/0.log" Jul 06 23:27:58.111338 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:58.111307 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fcc5d685f-k7msg_d70e9c41-fd49-4be6-91a4-a0f0f1fafba2/router/0.log" Jul 06 23:27:58.678598 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:27:58.678568 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-g6dv2_5accff7a-e5a6-475a-a221-7c85a71855f6/main/0.log" Jul 06 23:28:02.682616 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.682575 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h8t4j/must-gather-24l48"] Jul 06 23:28:02.683193 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683172 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" Jul 06 23:28:02.683278 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683195 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" Jul 06 23:28:02.683278 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683215 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" Jul 06 23:28:02.683278 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683225 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" Jul 06 23:28:02.683278 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683255 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" Jul 06 23:28:02.683278 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683264 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683279 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683288 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="36590ac4-67ba-4f0d-a7e2-5fbd511cda02" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683383 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683395 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683405 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c258e69-a044-44e0-83f4-cb353e803ebe" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683415 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683505 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" Jul 06 23:28:02.683531 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.683515 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa73f8b6-f717-48fe-b456-d95fa3224c7c" containerName="istio-proxy" Jul 06 23:28:02.688496 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.688473 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:02.691315 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.691289 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h8t4j\"/\"openshift-service-ca.crt\"" Jul 06 23:28:02.691439 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.691345 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h8t4j\"/\"kube-root-ca.crt\"" Jul 06 23:28:02.697209 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.697181 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h8t4j/must-gather-24l48"] Jul 06 23:28:02.827353 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.827302 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv8f7\" (UniqueName: \"kubernetes.io/projected/886d7e3a-b25e-4082-beef-cd146a59951d-kube-api-access-tv8f7\") pod \"must-gather-24l48\" (UID: \"886d7e3a-b25e-4082-beef-cd146a59951d\") " pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:02.827532 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.827427 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886d7e3a-b25e-4082-beef-cd146a59951d-must-gather-output\") pod \"must-gather-24l48\" (UID: \"886d7e3a-b25e-4082-beef-cd146a59951d\") " pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:02.928600 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.928561 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv8f7\" (UniqueName: \"kubernetes.io/projected/886d7e3a-b25e-4082-beef-cd146a59951d-kube-api-access-tv8f7\") pod \"must-gather-24l48\" (UID: \"886d7e3a-b25e-4082-beef-cd146a59951d\") " pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:02.928837 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.928643 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886d7e3a-b25e-4082-beef-cd146a59951d-must-gather-output\") pod \"must-gather-24l48\" (UID: \"886d7e3a-b25e-4082-beef-cd146a59951d\") " pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:02.929009 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.928986 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/886d7e3a-b25e-4082-beef-cd146a59951d-must-gather-output\") pod \"must-gather-24l48\" (UID: \"886d7e3a-b25e-4082-beef-cd146a59951d\") " pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:02.936375 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.936315 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv8f7\" (UniqueName: \"kubernetes.io/projected/886d7e3a-b25e-4082-beef-cd146a59951d-kube-api-access-tv8f7\") pod \"must-gather-24l48\" (UID: \"886d7e3a-b25e-4082-beef-cd146a59951d\") " pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:02.999303 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:02.999263 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h8t4j/must-gather-24l48" Jul 06 23:28:03.147955 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:03.147921 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h8t4j/must-gather-24l48"] Jul 06 23:28:03.148924 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:28:03.148891 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod886d7e3a_b25e_4082_beef_cd146a59951d.slice/crio-45eb21e769ec8bab5faf1fdeb2e3d7ac8d7aeab9019f5bbf84ee59e88c169c11 WatchSource:0}: Error finding container 45eb21e769ec8bab5faf1fdeb2e3d7ac8d7aeab9019f5bbf84ee59e88c169c11: Status 404 returned error can't find the container with id 45eb21e769ec8bab5faf1fdeb2e3d7ac8d7aeab9019f5bbf84ee59e88c169c11 Jul 06 23:28:03.150726 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:03.150704 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:28:03.380130 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:03.380037 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h8t4j/must-gather-24l48" event={"ID":"886d7e3a-b25e-4082-beef-cd146a59951d","Type":"ContainerStarted","Data":"45eb21e769ec8bab5faf1fdeb2e3d7ac8d7aeab9019f5bbf84ee59e88c169c11"} Jul 06 23:28:04.389052 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:04.388951 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h8t4j/must-gather-24l48" event={"ID":"886d7e3a-b25e-4082-beef-cd146a59951d","Type":"ContainerStarted","Data":"2bf1409e22b1bf57149e11acac9fc5059cd31edee3433c3f4bbd24a8321c328c"} Jul 06 23:28:04.389052 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:04.389002 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h8t4j/must-gather-24l48" event={"ID":"886d7e3a-b25e-4082-beef-cd146a59951d","Type":"ContainerStarted","Data":"30e946e6fb12b5149e3e8f95e72ebba43d674c5e432d176ed6d111933ac9fe90"} Jul 06 23:28:04.408761 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:04.408670 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h8t4j/must-gather-24l48" podStartSLOduration=1.492694438 podStartE2EDuration="2.408643214s" totalImagesPullingTime="915.948776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:28:04.40391142 +0000 UTC m=+2497.703613023" watchObservedRunningTime="2026-07-06 23:28:04.408643214 +0000 UTC m=+2497.708344818" Jul 06 23:28:05.655916 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:05.655878 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qsln2_fc293b72-c4f9-46bc-a0ae-ceb0f0cefe44/global-pull-secret-syncer/0.log" Jul 06 23:28:05.809327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:05.809292 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l5jvj_6e237774-91e5-4bbc-8cdb-cbc48de6d8d5/konnectivity-agent/0.log" Jul 06 23:28:05.830469 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:05.830436 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-38.ec2.internal_301ed78f06a8f545f44b18fb2b42ca58/haproxy/0.log" Jul 06 23:28:10.271158 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.271128 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx_d07fcd1e-5d05-4cec-a74c-5cf8464a0499/extract/0.log" Jul 06 23:28:10.293809 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.293783 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx_d07fcd1e-5d05-4cec-a74c-5cf8464a0499/util/0.log" Jul 06 23:28:10.321064 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.321034 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7597grhx_d07fcd1e-5d05-4cec-a74c-5cf8464a0499/pull/0.log" Jul 06 23:28:10.367124 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.367085 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj_6527ff3e-840b-47d3-9500-86dc480b51c3/extract/0.log" Jul 06 23:28:10.388618 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.388547 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj_6527ff3e-840b-47d3-9500-86dc480b51c3/util/0.log" Jul 06 23:28:10.412706 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.412664 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bglhj_6527ff3e-840b-47d3-9500-86dc480b51c3/pull/0.log" Jul 06 23:28:10.452727 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.452633 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2_7e30516f-e531-4072-a38b-67fe2f38ff4e/extract/0.log" Jul 06 23:28:10.477350 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.477201 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2_7e30516f-e531-4072-a38b-67fe2f38ff4e/util/0.log" Jul 06 23:28:10.501725 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.501674 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z6rr2_7e30516f-e531-4072-a38b-67fe2f38ff4e/pull/0.log" Jul 06 23:28:10.533077 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.532961 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb_7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187/extract/0.log" Jul 06 23:28:10.556081 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.556045 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb_7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187/util/0.log" Jul 06 23:28:10.582400 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.582353 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jk4kb_7d62bc7e-b2fc-4dd4-8b8f-9e4964c83187/pull/0.log" Jul 06 23:28:10.971427 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.971353 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-696b84895b-n86s9_d4d6b552-f397-4a8b-acb4-1edcb6483373/authorino/0.log" Jul 06 23:28:10.997599 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:10.997535 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qhhjt_3bfc6fc4-7482-46af-bf1d-cead4d5104b9/manager/0.log" Jul 06 23:28:11.057133 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:11.057098 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6n45n_0910a5f2-2bbb-49d8-8c37-ff472d59504e/kuadrant-console-plugin/0.log" Jul 06 23:28:11.099327 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:11.099282 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p74df_c9006499-1789-4d73-99dd-0f422c4c8a3f/registry-server/0.log" Jul 06 23:28:11.267052 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:11.266791 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-m2jjp_df9e3161-134d-47d7-a439-ac3bf278703a/manager/0.log" Jul 06 23:28:11.365709 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:11.365157 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-g4lpc_55e16c41-d39f-454b-8307-624b4925bdbe/manager/0.log" Jul 06 23:28:13.324639 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:13.323751 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-khkw7_b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c/node-exporter/0.log" Jul 06 23:28:13.351853 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:13.351620 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-khkw7_b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c/kube-rbac-proxy/0.log" Jul 06 23:28:13.375032 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:13.374996 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-khkw7_b8e39119-24e6-44c0-8cbc-f2f4c4c06d0c/init-textfile/0.log" Jul 06 23:28:15.308545 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:15.308513 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-jpkh8_77db57d6-78b8-43c9-9ce2-7e8e045f4377/networking-console-plugin/0.log" Jul 06 23:28:15.906804 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:15.906770 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/2.log" Jul 06 23:28:15.912005 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:15.911971 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pj9wr_18094565-1360-46e6-9800-d2e14b59c959/console-operator/3.log" Jul 06 23:28:16.215189 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.215152 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4"] Jul 06 23:28:16.223465 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.223429 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.226086 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.226052 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4"] Jul 06 23:28:16.226086 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.226079 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h8t4j\"/\"default-dockercfg-csq5b\"" Jul 06 23:28:16.291837 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.291797 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-proc\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.292180 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.292157 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-sys\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.292394 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.292379 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-lib-modules\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.292561 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.292543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-podres\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.292758 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.292742 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46k2\" (UniqueName: \"kubernetes.io/projected/63648651-e915-42ce-8aa0-bfc58c0ddd84-kube-api-access-r46k2\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394182 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-lib-modules\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394306 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-podres\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394337 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46k2\" (UniqueName: \"kubernetes.io/projected/63648651-e915-42ce-8aa0-bfc58c0ddd84-kube-api-access-r46k2\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394399 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-proc\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394478 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-sys\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394597 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-sys\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394746 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-lib-modules\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.394825 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-podres\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.395295 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.395243 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63648651-e915-42ce-8aa0-bfc58c0ddd84-proc\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.403870 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.403831 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46k2\" (UniqueName: \"kubernetes.io/projected/63648651-e915-42ce-8aa0-bfc58c0ddd84-kube-api-access-r46k2\") pod \"perf-node-gather-daemonset-vn9g4\" (UID: \"63648651-e915-42ce-8aa0-bfc58c0ddd84\") " pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.469593 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.469510 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7698c68448-dttmw_0fadf873-a021-48a6-9f5f-ee106affe381/console/0.log" Jul 06 23:28:16.538779 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.538509 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:16.748544 ip-10-0-128-38 kubenswrapper[2561]: W0706 23:28:16.748502 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod63648651_e915_42ce_8aa0_bfc58c0ddd84.slice/crio-2f4c8c0ce222ea29557768465701a644f9ccdfab4a826a4069d38331640ec369 WatchSource:0}: Error finding container 2f4c8c0ce222ea29557768465701a644f9ccdfab4a826a4069d38331640ec369: Status 404 returned error can't find the container with id 2f4c8c0ce222ea29557768465701a644f9ccdfab4a826a4069d38331640ec369 Jul 06 23:28:16.748676 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:16.748606 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4"] Jul 06 23:28:17.215165 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:17.215112 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-gq6q8_77391107-c4a1-4d4e-b602-21a6bfeb9c25/volume-data-source-validator/0.log" Jul 06 23:28:17.503314 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:17.502961 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" event={"ID":"63648651-e915-42ce-8aa0-bfc58c0ddd84","Type":"ContainerStarted","Data":"23167b5560282f20594d7995d2ea176b09036b32227095d17d39c8c2428a1f83"} Jul 06 23:28:17.503314 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:17.503289 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" event={"ID":"63648651-e915-42ce-8aa0-bfc58c0ddd84","Type":"ContainerStarted","Data":"2f4c8c0ce222ea29557768465701a644f9ccdfab4a826a4069d38331640ec369"} Jul 06 23:28:17.503959 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:17.503330 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:17.521965 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:17.521906 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" podStartSLOduration=1.52188421 podStartE2EDuration="1.52188421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:28:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:28:17.520720467 +0000 UTC m=+2510.820422093" watchObservedRunningTime="2026-07-06 23:28:17.52188421 +0000 UTC m=+2510.821585812" Jul 06 23:28:18.348589 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:18.348527 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8p8t2_4d3599fe-dbea-4bdc-b913-846b419c5fac/dns/0.log" Jul 06 23:28:18.369959 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:18.369933 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8p8t2_4d3599fe-dbea-4bdc-b913-846b419c5fac/kube-rbac-proxy/0.log" Jul 06 23:28:18.460658 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:18.460613 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lgv7m_4027f84a-d2a8-4d7e-bdcb-3c7ab7642bc3/dns-node-resolver/0.log" Jul 06 23:28:19.019965 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:19.019932 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-65c97fdb9c-875fz_f86563b7-0cc6-4b18-842d-66751fc0ecb9/registry/0.log" Jul 06 23:28:19.094435 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:19.094402 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qlk8d_3eb86e34-7ac4-49e6-910d-a04169181619/node-ca/0.log" Jul 06 23:28:20.036698 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:20.036631 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5bp7_cdeabf53-1e52-4feb-9cb4-dce852a82f64/istio-proxy/0.log" Jul 06 23:28:20.193944 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:20.193919 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54f95589f8-jqc4s_e20fa170-f482-45a3-ac02-391f1a736daf/kube-auth-proxy/0.log" Jul 06 23:28:20.224348 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:20.224315 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hqqzn_61fd0385-52bf-4df5-afec-e6405b79461e/istio-proxy/0.log" Jul 06 23:28:20.257679 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:20.257645 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hqqzn_61fd0385-52bf-4df5-afec-e6405b79461e/istio-proxy/1.log" Jul 06 23:28:20.347966 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:20.347886 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-b5tlr_3d3caabc-69e1-4913-94b9-c9ca44a92f3a/payload-processing/0.log" Jul 06 23:28:20.369573 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:20.369537 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fcc5d685f-k7msg_d70e9c41-fd49-4be6-91a4-a0f0f1fafba2/router/0.log" Jul 06 23:28:21.085561 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:21.085522 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wtsvg_efb1a3a3-9752-443b-834d-6ae1b3c0b448/serve-healthcheck-canary/0.log" Jul 06 23:28:21.541437 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:21.541401 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-9hzjz_22fb2b7b-2c50-42c5-96fd-b2325515019f/insights-operator/0.log" Jul 06 23:28:21.542109 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:21.542084 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-9hzjz_22fb2b7b-2c50-42c5-96fd-b2325515019f/insights-operator/1.log" Jul 06 23:28:21.631237 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:21.631206 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dz8m4_a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb/kube-rbac-proxy/0.log" Jul 06 23:28:21.651925 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:21.651886 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dz8m4_a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb/exporter/0.log" Jul 06 23:28:21.673886 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:21.673857 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dz8m4_a4dcb7e8-2f91-4e33-a0cc-12e9f929b7eb/extractor/0.log" Jul 06 23:28:23.523788 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:23.523758 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h8t4j/perf-node-gather-daemonset-vn9g4" Jul 06 23:28:24.076586 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:24.076546 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hpp57_2ebebcc1-626f-4f0d-a317-b1c8edbb4349/manager/0.log" Jul 06 23:28:24.200154 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:24.200119 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cbrc5_a1e21ab9-d7e9-4f0d-9a03-8699c11061b6/server/0.log" Jul 06 23:28:24.291992 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:24.291953 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wvqp8_650d898d-1738-4b98-b1df-7ce751e04ab8/manager/0.log" Jul 06 23:28:24.326204 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:24.326162 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-cs6d2_6c93e238-ab31-4458-8a69-b797e29aa41b/manager/0.log" Jul 06 23:28:24.362039 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:24.361954 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84df5575d5-cs6d2_6c93e238-ab31-4458-8a69-b797e29aa41b/copy-manifests/0.log" Jul 06 23:28:27.017730 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:27.017696 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5bp7_cdeabf53-1e52-4feb-9cb4-dce852a82f64/istio-proxy/0.log" Jul 06 23:28:27.152600 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:27.152566 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54f95589f8-jqc4s_e20fa170-f482-45a3-ac02-391f1a736daf/kube-auth-proxy/0.log" Jul 06 23:28:27.177610 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:27.177579 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hqqzn_61fd0385-52bf-4df5-afec-e6405b79461e/istio-proxy/0.log" Jul 06 23:28:27.212021 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:27.211981 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hqqzn_61fd0385-52bf-4df5-afec-e6405b79461e/istio-proxy/1.log" Jul 06 23:28:27.281774 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:27.281663 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-b5tlr_3d3caabc-69e1-4913-94b9-c9ca44a92f3a/payload-processing/0.log" Jul 06 23:28:27.297912 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:27.297878 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fcc5d685f-k7msg_d70e9c41-fd49-4be6-91a4-a0f0f1fafba2/router/0.log" Jul 06 23:28:28.517764 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:28.517736 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678d5bp7_cdeabf53-1e52-4feb-9cb4-dce852a82f64/istio-proxy/0.log" Jul 06 23:28:28.649885 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:28.649851 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54f95589f8-jqc4s_e20fa170-f482-45a3-ac02-391f1a736daf/kube-auth-proxy/0.log" Jul 06 23:28:28.673550 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:28.673517 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hqqzn_61fd0385-52bf-4df5-afec-e6405b79461e/istio-proxy/0.log" Jul 06 23:28:28.708267 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:28.708235 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hqqzn_61fd0385-52bf-4df5-afec-e6405b79461e/istio-proxy/1.log" Jul 06 23:28:28.782092 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:28.782008 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-b5tlr_3d3caabc-69e1-4913-94b9-c9ca44a92f3a/payload-processing/0.log" Jul 06 23:28:28.798521 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:28.798491 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fcc5d685f-k7msg_d70e9c41-fd49-4be6-91a4-a0f0f1fafba2/router/0.log" Jul 06 23:28:32.690060 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:32.690032 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bw8fc_ad332d8f-fbd9-4a1d-8ff0-3473e9d50590/kube-multus-additional-cni-plugins/0.log" Jul 06 23:28:32.711773 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:32.711743 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bw8fc_ad332d8f-fbd9-4a1d-8ff0-3473e9d50590/egress-router-binary-copy/0.log" Jul 06 23:28:32.731833 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:32.731795 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bw8fc_ad332d8f-fbd9-4a1d-8ff0-3473e9d50590/cni-plugins/0.log" Jul 06 23:28:32.751166 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:32.751139 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bw8fc_ad332d8f-fbd9-4a1d-8ff0-3473e9d50590/bond-cni-plugin/0.log" Jul 06 23:28:32.770578 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:32.770548 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bw8fc_ad332d8f-fbd9-4a1d-8ff0-3473e9d50590/routeoverride-cni/0.log" Jul 06 23:28:32.789938 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:32.789907 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bw8fc_ad332d8f-fbd9-4a1d-8ff0-3473e9d50590/whereabouts-cni-bincopy/0.log" Jul 06 23:28:32.809303 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:32.809270 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bw8fc_ad332d8f-fbd9-4a1d-8ff0-3473e9d50590/whereabouts-cni/0.log" Jul 06 23:28:33.139130 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:33.139060 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pdb5z_7282657b-60b7-4d5f-976c-306625667e03/kube-multus/0.log" Jul 06 23:28:33.233177 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:33.233145 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-82jtz_5422cceb-c1fe-4581-847a-59883489e159/network-metrics-daemon/0.log" Jul 06 23:28:33.251610 ip-10-0-128-38 kubenswrapper[2561]: I0706 23:28:33.251580 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-82jtz_5422cceb-c1fe-4581-847a-59883489e159/kube-rbac-proxy/0.log"