Jul 07 02:09:43.120457 ip-10-0-141-30 systemd[1]: Starting Kubernetes Kubelet... Jul 07 02:09:43.587269 ip-10-0-141-30 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 02:09:43.587269 ip-10-0-141-30 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 02:09:43.587269 ip-10-0-141-30 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 02:09:43.587269 ip-10-0-141-30 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 02:09:43.587269 ip-10-0-141-30 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 02:09:43.590276 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.590124 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 02:09:43.596141 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596123 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 02:09:43.596141 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596141 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596147 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596163 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596167 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596171 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596174 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596177 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596180 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596183 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596185 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596188 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596191 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596194 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596196 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596199 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596202 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596204 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596207 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596211 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 02:09:43.596217 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596215 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596218 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596222 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596225 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596228 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596232 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596234 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596237 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596246 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596249 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596251 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596255 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596258 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596261 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596263 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596266 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596268 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596271 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596273 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596276 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 02:09:43.596683 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596278 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596281 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596283 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596286 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596288 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596291 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596293 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596295 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596298 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596300 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596304 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596307 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596310 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596312 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596315 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596318 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596320 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596323 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596326 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596328 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 02:09:43.597165 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596331 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596333 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596336 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596338 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596344 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596346 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596349 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596351 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596354 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596356 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596359 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596363 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596366 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596368 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596372 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596376 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596379 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596382 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596385 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 02:09:43.597653 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596387 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596390 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596393 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596396 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596399 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596402 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596405 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596408 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596411 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596414 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596417 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596420 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596422 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596425 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596427 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596430 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596433 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596435 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596440 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 02:09:43.598092 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.596443 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597163 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597171 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597174 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597177 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597181 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597184 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597187 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597191 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597193 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597196 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597199 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597201 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597204 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597207 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597209 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597212 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597215 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597226 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597229 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 02:09:43.598545 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597232 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597235 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597238 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597240 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597243 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597246 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597248 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597251 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597254 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597257 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597259 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597262 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597265 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597268 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597271 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597273 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597276 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597279 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597282 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597285 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 02:09:43.599062 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597287 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597290 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597293 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597295 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597298 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597300 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597303 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597305 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597308 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597310 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597313 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597316 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597318 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597323 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597327 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597330 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597332 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597335 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597338 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597340 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 02:09:43.599551 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597343 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597346 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597348 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597351 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597353 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597356 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597359 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597362 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597364 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597367 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597370 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597373 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597375 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597378 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597380 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597384 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597388 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597391 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597393 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 02:09:43.600027 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597396 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597398 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597401 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597404 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597406 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597410 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597412 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597415 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597418 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597420 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597423 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597425 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597428 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597431 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597434 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597436 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597439 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597445 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597449 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 02:09:43.600502 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597452 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597455 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597852 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597857 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597861 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597864 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597867 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597870 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597873 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597876 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597879 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597882 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597884 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597887 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597889 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597892 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597895 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597897 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597900 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597902 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 02:09:43.600962 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597905 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597908 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597910 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597913 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597915 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597918 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597920 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597923 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597926 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597929 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597935 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597938 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597941 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597944 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597947 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597951 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597966 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597970 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597973 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 02:09:43.601465 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597976 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597979 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597983 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597986 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597989 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597991 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597994 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.597997 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598000 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598003 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598006 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598009 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598012 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598015 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598018 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598021 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598023 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598026 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598029 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598031 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 02:09:43.602028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598034 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598036 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598039 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598045 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598048 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598051 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598055 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598059 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598062 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598065 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598067 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598070 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598073 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598076 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598079 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598081 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598084 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598086 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598089 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 02:09:43.602531 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598102 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598106 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598109 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598112 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598114 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598117 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598120 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598123 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598125 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598128 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598131 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598133 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598135 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598138 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598141 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598143 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598146 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598149 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598165 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598168 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 02:09:43.602975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598171 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598173 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598176 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598238 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598245 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598251 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598256 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598260 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598263 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598267 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598271 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598274 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598277 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598281 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598284 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598287 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598290 2559 flags.go:64] FLAG: --cgroup-root="" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598293 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598296 2559 flags.go:64] FLAG: --client-ca-file="" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598299 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598302 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598306 2559 flags.go:64] FLAG: --cluster-domain="" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598309 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 02:09:43.603464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598312 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598315 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598318 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598322 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598325 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598329 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598332 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598335 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598338 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598341 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598345 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598348 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598352 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598355 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598358 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598361 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598364 2559 flags.go:64] FLAG: --enable-server="true" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598367 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598372 2559 flags.go:64] FLAG: --event-burst="100" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598375 2559 flags.go:64] FLAG: --event-qps="50" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598378 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598381 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598384 2559 flags.go:64] FLAG: --eviction-hard="" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598388 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 02:09:43.604004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598391 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598394 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598397 2559 flags.go:64] FLAG: --eviction-soft="" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598401 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598404 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598407 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598411 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598414 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598417 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598419 2559 flags.go:64] FLAG: --feature-gates="" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598423 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598426 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598430 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598434 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598437 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598440 2559 flags.go:64] FLAG: --help="false" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598443 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598446 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598450 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598453 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598457 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598460 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598463 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 02:09:43.604610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598466 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598469 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598472 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598474 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598477 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598480 2559 flags.go:64] FLAG: --kube-reserved="" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598483 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598486 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598489 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598492 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598494 2559 flags.go:64] FLAG: --lock-file="" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598497 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598501 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598504 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598510 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598513 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598516 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598518 2559 flags.go:64] FLAG: --logging-format="text" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598521 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598525 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598527 2559 flags.go:64] FLAG: --manifest-url="" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598530 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598534 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598537 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598541 2559 flags.go:64] FLAG: --max-pods="110" Jul 07 02:09:43.605195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598544 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598547 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598550 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598553 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598557 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598560 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598563 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598569 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598572 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598575 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598578 2559 flags.go:64] FLAG: --pod-cidr="" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598582 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598587 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598590 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598593 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598596 2559 flags.go:64] FLAG: --port="10250" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598599 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598602 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f50df80ce97e663e" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598605 2559 flags.go:64] FLAG: --qos-reserved="" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598609 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598612 2559 flags.go:64] FLAG: --register-node="true" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598615 2559 flags.go:64] FLAG: --register-with-taints="" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598618 2559 flags.go:64] FLAG: --registry-burst="10" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598621 2559 flags.go:64] FLAG: --registry-qps="5" Jul 07 02:09:43.605798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598624 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598627 2559 flags.go:64] FLAG: --reserved-memory="" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598632 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598635 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598638 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598640 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598643 2559 flags.go:64] FLAG: --runonce="false" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598646 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598649 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598652 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598655 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598658 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598661 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598665 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598668 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598671 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598674 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598677 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598680 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598683 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598686 2559 flags.go:64] FLAG: --system-cgroups="" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598688 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598693 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598696 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598699 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 02:09:43.606438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598703 2559 flags.go:64] FLAG: --tls-min-version="" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598706 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598709 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598712 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598715 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598718 2559 flags.go:64] FLAG: --v="2" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598722 2559 flags.go:64] FLAG: --version="false" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598726 2559 flags.go:64] FLAG: --vmodule="" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598729 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.598734 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598844 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598849 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598852 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598855 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598858 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598861 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598864 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598866 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598869 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598872 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598876 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 02:09:43.607472 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598880 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598883 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598886 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598889 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598892 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598895 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598898 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598901 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598903 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598906 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598909 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598911 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598914 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598918 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598920 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598923 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598926 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598928 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598931 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598934 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 02:09:43.607988 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598938 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598941 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598945 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598949 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598952 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598954 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598957 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598960 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598963 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598965 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598968 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598970 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598973 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598980 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598982 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598985 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598987 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598990 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598993 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 02:09:43.608529 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.598996 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599000 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599002 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599005 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599008 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599011 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599018 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599020 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599023 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599025 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599028 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599030 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599033 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599037 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599040 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599042 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599045 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599048 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599051 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599053 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 02:09:43.608975 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599056 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599059 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599061 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599064 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599067 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599070 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599073 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599076 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599079 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599081 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599084 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599086 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599089 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599091 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599094 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599097 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599099 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599102 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599106 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599108 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 02:09:43.609480 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599111 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599114 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599116 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599119 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599121 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599126 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599129 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599131 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.599134 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.600176 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.608421 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.608437 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.608457 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.608463 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 02:09:43.609949 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608520 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608525 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608528 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608532 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608535 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608539 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608542 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608547 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608551 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608554 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608557 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608559 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608562 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608565 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608568 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608571 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608573 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608576 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608579 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608581 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 02:09:43.610310 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608585 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608588 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608591 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608594 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608596 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608599 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608601 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608604 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608607 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608611 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608613 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608616 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608619 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608622 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608625 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608627 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608630 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608633 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608635 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608638 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 02:09:43.610780 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608640 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608644 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608648 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608650 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608653 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608656 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608659 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608662 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608664 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608667 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608669 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608672 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608675 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608677 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608680 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608683 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608685 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608688 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608691 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608693 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 02:09:43.611272 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608696 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608699 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608702 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608705 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608707 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608710 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608713 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608715 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608718 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608720 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608723 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608726 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608728 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608731 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608734 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608737 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608739 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608742 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608745 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608748 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 02:09:43.611746 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608750 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608753 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608756 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608758 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608761 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608764 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608766 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608769 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608772 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608774 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608777 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608780 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608782 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608785 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608787 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608790 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608793 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608796 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 02:09:43.612240 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608800 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.608805 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608936 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608941 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608945 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608948 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608951 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608953 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608956 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608958 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608961 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608964 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608966 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608969 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608972 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 02:09:43.612665 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608975 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608978 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608981 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608983 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608986 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608989 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608992 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608995 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.608997 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609000 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609002 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609005 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609007 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609010 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609013 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609015 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609018 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609021 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609023 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609026 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 02:09:43.613023 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609028 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609031 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609033 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609036 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609038 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609041 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609043 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609046 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609048 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609051 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609053 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609056 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609059 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609062 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609065 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609067 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609070 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609073 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609075 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609078 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 02:09:43.613556 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609080 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609083 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609087 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609091 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609093 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609096 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609099 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609101 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609104 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609106 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609109 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609111 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609114 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609116 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609119 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609122 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609124 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609126 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609129 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 02:09:43.614028 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609131 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609134 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609137 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609139 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609142 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609146 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609165 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609170 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609173 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609176 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609178 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609181 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609184 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609186 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609189 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609191 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609194 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609196 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609201 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 02:09:43.614488 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609204 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609207 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609210 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609213 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609223 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609227 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609230 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:43.609232 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.609237 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.609353 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.612964 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.614588 2559 server.go:1019] "Starting client certificate rotation" Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.614688 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 02:09:43.614933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.614726 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 02:09:43.617237 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.617213 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 02:09:43.618414 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.618399 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 02:09:43.625966 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.625950 2559 log.go:25] "Validated CRI v1 runtime API" Jul 07 02:09:43.628230 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.628218 2559 log.go:25] "Validated CRI v1 image API" Jul 07 02:09:43.629430 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.629417 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 02:09:43.633615 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.633597 2559 fs.go:135] Filesystem UUIDs: map[481ffa09-fe04-42dc-b24b-3dc49e31fa1d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c60093c8-6e8e-47ba-a8a7-f1756c9e76ff:/dev/nvme0n1p3] Jul 07 02:09:43.633666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.633615 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 02:09:43.639557 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.639454 2559 manager.go:216] Machine: {Timestamp:2026-07-07 02:09:43.638955511 +0000 UTC m=+0.389377155 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3119832 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2626036903fdd2b978a86124e029ab SystemUUID:ec262603-6903-fdd2-b978-a86124e029ab BootID:9f78bcb8-4bf8-4cf0-bbca-481678f9ebb4 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b3:10:0a:6b:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b3:10:0a:6b:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:f9:32:a2:24:73 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 02:09:43.639557 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.639549 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 02:09:43.639678 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.639621 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 02:09:43.642205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.642181 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 02:09:43.642333 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.642208 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-30.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 02:09:43.643356 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.643344 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 02:09:43.643356 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.643356 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 02:09:43.643441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.643364 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 02:09:43.643954 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.643944 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 02:09:43.644700 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.644679 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 02:09:43.645626 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.645614 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 02:09:43.648829 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.648818 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 07 02:09:43.648933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.648924 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 02:09:43.648971 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.648951 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 07 02:09:43.648971 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.648960 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 02:09:43.649028 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.648974 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 02:09:43.649028 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.648982 2559 kubelet.go:398] "Adding apiserver pod source" Jul 07 02:09:43.649028 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.648992 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 02:09:43.649404 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.649392 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 02:09:43.649445 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.649411 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 02:09:43.652892 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.652876 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 02:09:43.655343 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.655330 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 02:09:43.655390 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.655359 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 02:09:43.656532 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656521 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656537 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656543 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656549 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656557 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656563 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656571 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656576 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 02:09:43.656580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656583 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 02:09:43.656824 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656589 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 02:09:43.656824 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656597 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 02:09:43.656824 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.656604 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 02:09:43.658682 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.658669 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 02:09:43.658682 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.658682 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 02:09:43.659705 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.659682 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 02:09:43.659769 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.659709 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 02:09:43.661688 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.661676 2559 server.go:1268] "Started kubelet" Jul 07 02:09:43.661832 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.661783 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 02:09:43.661871 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.661856 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 02:09:43.662485 ip-10-0-141-30 systemd[1]: Started Kubernetes Kubelet. Jul 07 02:09:43.662756 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.662728 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 02:09:43.663791 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.663775 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 02:09:43.665745 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.665731 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 07 02:09:43.666407 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.666378 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ltg9z" Jul 07 02:09:43.668238 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.668219 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-30.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 02:09:43.668745 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.668727 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 02:09:43.669190 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.667981 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-30.ec2.internal.18bfdfec59123dd7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-30.ec2.internal,UID:ip-10-0-141-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-30.ec2.internal,},FirstTimestamp:2026-07-07 02:09:43.661649367 +0000 UTC m=+0.412071012,LastTimestamp:2026-07-07 02:09:43.661649367 +0000 UTC m=+0.412071012,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-30.ec2.internal,}" Jul 07 02:09:43.673870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.673845 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ltg9z" Jul 07 02:09:43.674716 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.674702 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 02:09:43.674796 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.674770 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 02:09:43.675120 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.675091 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:43.675748 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.675727 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 02:09:43.675748 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.675749 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 02:09:43.675870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.675752 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 02:09:43.675870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.675861 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 07 02:09:43.675870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.675870 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 07 02:09:43.676132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.676103 2559 factory.go:55] Registering systemd factory Jul 07 02:09:43.676132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.676125 2559 factory.go:223] Registration of the systemd container factory successfully Jul 07 02:09:43.676665 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.676647 2559 factory.go:153] Registering CRI-O factory Jul 07 02:09:43.676665 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.676668 2559 factory.go:223] Registration of the crio container factory successfully Jul 07 02:09:43.676825 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.676722 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 02:09:43.676825 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.676745 2559 factory.go:103] Registering Raw factory Jul 07 02:09:43.676825 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.676760 2559 manager.go:1196] Started watching for new ooms in manager Jul 07 02:09:43.676971 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.676855 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 02:09:43.676971 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.676877 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 02:09:43.677314 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.677296 2559 manager.go:349] Starting recovery of all containers Jul 07 02:09:43.687032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.686896 2559 manager.go:354] Recovery completed Jul 07 02:09:43.690928 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.690916 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 02:09:43.692759 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.692743 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientMemory" Jul 07 02:09:43.692821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.692773 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 02:09:43.692821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.692783 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientPID" Jul 07 02:09:43.693398 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.693383 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 02:09:43.693398 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.693395 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 02:09:43.693545 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.693410 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 07 02:09:43.693545 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.693427 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 02:09:43.694075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.694062 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientMemory" Jul 07 02:09:43.694139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.694092 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 02:09:43.694139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.694109 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientPID" Jul 07 02:09:43.695001 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.694988 2559 policy_none.go:49] "None policy: Start" Jul 07 02:09:43.695065 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.695004 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 02:09:43.695065 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.695014 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 02:09:43.696512 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.696501 2559 policy_none.go:47] "Start" Jul 07 02:09:43.734105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.734091 2559 manager.go:337] "Starting Device Plugin manager" Jul 07 02:09:43.734191 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.734119 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 02:09:43.734191 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.734129 2559 server.go:87] "Starting device plugin registration server" Jul 07 02:09:43.734686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.734661 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 02:09:43.734778 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.734706 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 02:09:43.734985 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.734970 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 02:09:43.735057 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.735047 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 02:09:43.735092 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.735057 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 02:09:43.736842 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.735972 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 02:09:43.736842 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.736029 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:43.810585 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.810533 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 02:09:43.811785 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.811767 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 02:09:43.811837 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.811821 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 02:09:43.811869 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.811848 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 02:09:43.811909 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.811891 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 02:09:43.814407 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.814385 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 02:09:43.835210 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.835188 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 02:09:43.836042 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.836022 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientMemory" Jul 07 02:09:43.836130 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.836055 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 02:09:43.836130 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.836065 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientPID" Jul 07 02:09:43.836130 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.836089 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.841800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.841757 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.841800 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.841779 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-30.ec2.internal\": node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:43.854307 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.854286 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:43.912749 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.912713 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal"] Jul 07 02:09:43.912930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.912839 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 02:09:43.914581 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.914554 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientMemory" Jul 07 02:09:43.914581 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.914588 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 02:09:43.914728 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.914598 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientPID" Jul 07 02:09:43.916003 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.915992 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 02:09:43.916123 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916109 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.916185 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916140 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 02:09:43.916682 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916668 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientMemory" Jul 07 02:09:43.916750 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916688 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 02:09:43.916750 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916698 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientPID" Jul 07 02:09:43.916750 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916670 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientMemory" Jul 07 02:09:43.916837 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916758 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 02:09:43.916837 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.916769 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientPID" Jul 07 02:09:43.918026 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.918010 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.918085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.918036 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 02:09:43.918629 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.918614 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientMemory" Jul 07 02:09:43.918697 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.918644 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 02:09:43.918697 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.918655 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeHasSufficientPID" Jul 07 02:09:43.939760 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.939733 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-30.ec2.internal\" not found" node="ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.944095 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.944079 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-30.ec2.internal\" not found" node="ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.954599 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:43.954582 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:43.977013 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.976993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/419e48f62642a0af69b25fe0fa0d0553-config\") pod \"kube-apiserver-proxy-ip-10-0-141-30.ec2.internal\" (UID: \"419e48f62642a0af69b25fe0fa0d0553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.977075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.977019 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c119a6373627d321976d8403f321294-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal\" (UID: \"1c119a6373627d321976d8403f321294\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:43.977075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:43.977036 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c119a6373627d321976d8403f321294-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal\" (UID: \"1c119a6373627d321976d8403f321294\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.054984 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.054942 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.077253 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.077223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c119a6373627d321976d8403f321294-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal\" (UID: \"1c119a6373627d321976d8403f321294\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.077319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.077265 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/419e48f62642a0af69b25fe0fa0d0553-config\") pod \"kube-apiserver-proxy-ip-10-0-141-30.ec2.internal\" (UID: \"419e48f62642a0af69b25fe0fa0d0553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.077319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.077284 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c119a6373627d321976d8403f321294-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal\" (UID: \"1c119a6373627d321976d8403f321294\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.077380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.077325 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c119a6373627d321976d8403f321294-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal\" (UID: \"1c119a6373627d321976d8403f321294\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.077380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.077333 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/419e48f62642a0af69b25fe0fa0d0553-config\") pod \"kube-apiserver-proxy-ip-10-0-141-30.ec2.internal\" (UID: \"419e48f62642a0af69b25fe0fa0d0553\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.077380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.077333 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c119a6373627d321976d8403f321294-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal\" (UID: \"1c119a6373627d321976d8403f321294\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.155600 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.155570 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.244054 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.244029 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.248427 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.248216 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" Jul 07 02:09:44.255954 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.255939 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.356435 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.356404 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.456986 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.456913 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.557403 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.557370 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.613893 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.613864 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 02:09:44.614376 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.614029 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 02:09:44.658274 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.658246 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.676055 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.676024 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 02:04:43 +0000 UTC" deadline="2027-12-19 05:05:03.139361654 +0000 UTC" Jul 07 02:09:44.676055 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.676047 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12722h55m18.463317632s" Jul 07 02:09:44.759202 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.759150 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.814610 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.814578 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 02:09:44.835088 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:44.835062 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c119a6373627d321976d8403f321294.slice/crio-b09b129f2cc95a756c790a39b964289bfba12138dd49aca56f410a43322c56ff WatchSource:0}: Error finding container b09b129f2cc95a756c790a39b964289bfba12138dd49aca56f410a43322c56ff: Status 404 returned error can't find the container with id b09b129f2cc95a756c790a39b964289bfba12138dd49aca56f410a43322c56ff Jul 07 02:09:44.835444 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:44.835424 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419e48f62642a0af69b25fe0fa0d0553.slice/crio-41e561a0b0726930c12fcc68f847c0774cfd7d9d8bfd8010095c24f064a15e80 WatchSource:0}: Error finding container 41e561a0b0726930c12fcc68f847c0774cfd7d9d8bfd8010095c24f064a15e80: Status 404 returned error can't find the container with id 41e561a0b0726930c12fcc68f847c0774cfd7d9d8bfd8010095c24f064a15e80 Jul 07 02:09:44.841383 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.841309 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 02:09:44.859718 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.859695 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.960211 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:44.960178 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:44.996594 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:44.996528 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 02:09:45.060495 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.060462 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-30.ec2.internal\" not found" Jul 07 02:09:45.099350 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.099330 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 02:09:45.175602 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.175574 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" Jul 07 02:09:45.189129 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.189111 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 02:09:45.190335 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.190322 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" Jul 07 02:09:45.196304 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.196283 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 02:09:45.650639 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.650607 2559 apiserver.go:52] "Watching apiserver" Jul 07 02:09:45.657487 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.657467 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 02:09:45.659941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.659909 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal","openshift-cluster-node-tuning-operator/tuned-j87hz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal","openshift-multus/multus-additional-cni-plugins-hhh5n","openshift-multus/multus-dzhkk","openshift-multus/network-metrics-daemon-44l58","openshift-network-diagnostics/network-check-target-qwwxl","kube-system/konnectivity-agent-wvvwk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb","openshift-dns/node-resolver-dcf4t","openshift-image-registry/node-ca-b4gg5","openshift-network-operator/iptables-alerter-jnndx","openshift-ovn-kubernetes/ovnkube-node-9c6j6","kube-system/global-pull-secret-syncer-d75g5"] Jul 07 02:09:45.661815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.661795 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:45.661939 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.661876 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:45.665036 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.664242 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.667805 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.666562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.667805 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.666698 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:09:45.668081 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.668061 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.668254 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.668226 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 02:09:45.668254 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.668243 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 02:09:45.668389 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.668299 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 02:09:45.668595 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.668578 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 02:09:45.669079 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.668749 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 02:09:45.669079 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.668901 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 02:09:45.669532 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.669279 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v2zzd\"" Jul 07 02:09:45.670308 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.669927 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.672062 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.671861 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 02:09:45.672179 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.672089 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 02:09:45.672634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.672252 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 02:09:45.672634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.672458 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 02:09:45.672634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.672475 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 02:09:45.672858 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.672801 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-76nzr\"" Jul 07 02:09:45.673213 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.673104 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 02:09:45.673213 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.673177 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-phwsk\"" Jul 07 02:09:45.673465 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.673454 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:45.673545 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.673507 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:45.675456 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.675438 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.676210 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.676188 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 02:09:45.677342 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.677197 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:45.678382 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.678365 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 02:09:45.678517 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.678388 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 02:09:45.678693 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.678444 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kkhjl\"" Jul 07 02:09:45.678772 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.678452 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 02:09:45.678828 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.678593 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.679267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.679249 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 02:09:45.679467 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.679451 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tl5d4\"" Jul 07 02:09:45.679696 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.679679 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 02:09:45.680282 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.680267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.681228 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.681211 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tn7hn\"" Jul 07 02:09:45.681487 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.681442 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 02:09:45.681910 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.681841 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.682261 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.682244 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 02:09:45.682752 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.682731 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cj662\"" Jul 07 02:09:45.682856 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.682807 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 02:09:45.682966 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.682755 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 02:09:45.683603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.683579 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.684139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.684120 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tw4wg\"" Jul 07 02:09:45.685295 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.684947 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 02:09:45.685295 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.684975 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 02:09:45.685295 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.684954 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 02:09:45.685508 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685454 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-systemd\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.685508 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-cnibin\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.685608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685504 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5plr\" (UniqueName: \"kubernetes.io/projected/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-kube-api-access-t5plr\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.685608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685531 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cni-binary-copy\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.685608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685555 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a395fe49-f597-4ebe-bf24-38af015944f6-multus-daemon-config\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.685608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685578 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.685608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685602 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685623 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-hostroot\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685649 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-etc-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-os-release\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685720 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685743 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-socket-dir-parent\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685769 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-cni-bin\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685785 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b459b352-3f73-4a90-a434-440bcb843220-dbus\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685799 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-system-cni-dir\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685815 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a395fe49-f597-4ebe-bf24-38af015944f6-cni-binary-copy\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685836 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:45.685864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685861 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535498bf-bd89-46dd-9937-2f3adce9941b-konnectivity-ca\") pod \"konnectivity-agent-wvvwk\" (UID: \"535498bf-bd89-46dd-9937-2f3adce9941b\") " pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685892 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-ovn\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685917 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-ovnkube-script-lib\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685944 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-system-cni-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.685988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-netns\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686042 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tf96\" (UniqueName: \"kubernetes.io/projected/af9fae31-f4c3-4823-b486-19c0322111bb-kube-api-access-4tf96\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686068 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-systemd-units\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686090 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cnibin\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686112 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-node-log\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686173 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-log-socket\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686196 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-cni-bin\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686266 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfq2h\" (UniqueName: \"kubernetes.io/projected/b0278ab3-1cd5-4503-8554-43f8bc4c8398-kube-api-access-gfq2h\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-cni-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-os-release\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.686448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686341 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-slash\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686391 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-ovnkube-config\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-multus-certs\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-device-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686466 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-sys-fs\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686474 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686515 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686645 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b459b352-3f73-4a90-a434-440bcb843220-kubelet-config\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686690 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686694 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-etc-kubernetes\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686767 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686791 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx28x\" (UniqueName: \"kubernetes.io/projected/b59a8c49-adcb-4435-8667-41a2203d706c-kube-api-access-hx28x\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686832 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-kubelet\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-run-netns\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-cni-netd\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686905 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-kubelet\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686972 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-env-overrides\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.686998 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b59a8c49-adcb-4435-8667-41a2203d706c-ovn-node-metrics-cert\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687003 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687021 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-cni-multus\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687047 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6f8d\" (UniqueName: \"kubernetes.io/projected/a395fe49-f597-4ebe-bf24-38af015944f6-kube-api-access-h6f8d\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687069 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-socket-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687093 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535498bf-bd89-46dd-9937-2f3adce9941b-agent-certs\") pod \"konnectivity-agent-wvvwk\" (UID: \"535498bf-bd89-46dd-9937-2f3adce9941b\") " pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687117 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-var-lib-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687140 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-registration-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687233 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-etc-selinux\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687259 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-k8s-cni-cncf-io\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687287 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f5j5d\"" Jul 07 02:09:45.687936 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.687290 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-conf-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.689545 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.689525 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 02:09:45.710984 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.710933 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kxfjm" Jul 07 02:09:45.718731 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.718529 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kxfjm" Jul 07 02:09:45.726860 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.726836 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 02:09:45.777523 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.777498 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 02:09:45.787870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.787843 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.787997 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.787884 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.787997 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.787910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-hostroot\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.787997 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.787967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-etc-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.788114 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-os-release\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.788114 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.788114 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.788037 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:45.788114 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788052 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-socket-dir-parent\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.788114 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788058 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-etc-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.788114 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788075 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-cni-bin\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.788114 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.788107 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret podName:b459b352-3f73-4a90-a434-440bcb843220 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:46.288085535 +0000 UTC m=+3.038507188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret") pod "global-pull-secret-syncer-d75g5" (UID: "b459b352-3f73-4a90-a434-440bcb843220") : object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788114 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-cni-bin\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788127 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-run\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788172 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b459b352-3f73-4a90-a434-440bcb843220-dbus\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788198 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-system-cni-dir\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788220 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a395fe49-f597-4ebe-bf24-38af015944f6-cni-binary-copy\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788265 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1b244eb-e66a-4f31-b82d-3ba053af3335-tmp\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788199 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-os-release\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788128 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-hostroot\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788323 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-socket-dir-parent\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788334 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788344 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-system-cni-dir\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535498bf-bd89-46dd-9937-2f3adce9941b-konnectivity-ca\") pod \"konnectivity-agent-wvvwk\" (UID: \"535498bf-bd89-46dd-9937-2f3adce9941b\") " pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.788357 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788405 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-ovn\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.788424 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:09:46.288409623 +0000 UTC m=+3.038831256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:45.788483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788451 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-ovnkube-script-lib\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788455 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b459b352-3f73-4a90-a434-440bcb843220-dbus\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-ovn\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788496 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-system-cni-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788508 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-netns\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788574 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tf96\" (UniqueName: \"kubernetes.io/projected/af9fae31-f4c3-4823-b486-19c0322111bb-kube-api-access-4tf96\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788582 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-system-cni-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788601 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-modprobe-d\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788645 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6684a217-290d-489a-8514-9b48a7940479-host\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788696 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-netns\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788712 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-systemd-units\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788750 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cnibin\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788788 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cnibin\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788789 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788804 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-systemd-units\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788843 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.789269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788869 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysctl-conf\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788886 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3f32190-7a1c-439a-a3b6-c6fa26c74542-iptables-alerter-script\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788905 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/535498bf-bd89-46dd-9937-2f3adce9941b-konnectivity-ca\") pod \"konnectivity-agent-wvvwk\" (UID: \"535498bf-bd89-46dd-9937-2f3adce9941b\") " pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-node-log\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788938 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-node-log\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788955 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-log-socket\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.788981 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-cni-bin\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789010 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-log-socket\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789020 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a395fe49-f597-4ebe-bf24-38af015944f6-cni-binary-copy\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789022 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-ovnkube-script-lib\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789011 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789055 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-cni-bin\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789080 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfq2h\" (UniqueName: \"kubernetes.io/projected/b0278ab3-1cd5-4503-8554-43f8bc4c8398-kube-api-access-gfq2h\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789106 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-cni-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789129 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-os-release\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789171 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789188 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-slash\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789202 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-ovnkube-config\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789216 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-multus-certs\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789237 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-device-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789260 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-sys-fs\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789276 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-sys\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789297 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-os-release\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789298 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-lib-modules\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789348 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789381 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24s58\" (UniqueName: \"kubernetes.io/projected/a1b244eb-e66a-4f31-b82d-3ba053af3335-kube-api-access-24s58\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b459b352-3f73-4a90-a434-440bcb843220-kubelet-config\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-etc-kubernetes\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789441 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-host\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-cni-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3f32190-7a1c-439a-a3b6-c6fa26c74542-host-slash\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789537 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b459b352-3f73-4a90-a434-440bcb843220-kubelet-config\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789543 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-device-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.790930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789570 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-multus-certs\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789552 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-sys-fs\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789584 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-etc-kubernetes\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789608 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789625 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789640 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-slash\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789645 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-ovnkube-config\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789649 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789685 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789718 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx28x\" (UniqueName: \"kubernetes.io/projected/b59a8c49-adcb-4435-8667-41a2203d706c-kube-api-access-hx28x\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6dd64963-deab-4e30-973a-d6cebc6e0209-hosts-file\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789773 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-kubelet\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789797 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-run-netns\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-cni-netd\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789840 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-kubelet\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789866 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dd64963-deab-4e30-973a-d6cebc6e0209-tmp-dir\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.791714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-env-overrides\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789912 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b59a8c49-adcb-4435-8667-41a2203d706c-ovn-node-metrics-cert\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789936 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-cni-netd\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789937 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-cni-multus\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789974 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-cni-multus\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.789974 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6f8d\" (UniqueName: \"kubernetes.io/projected/a395fe49-f597-4ebe-bf24-38af015944f6-kube-api-access-h6f8d\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790015 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-socket-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790027 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-var-lib-kubelet\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-tuned\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790105 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-kubelet\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790148 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-host-run-netns\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790415 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b59a8c49-adcb-4435-8667-41a2203d706c-env-overrides\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790469 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535498bf-bd89-46dd-9937-2f3adce9941b-agent-certs\") pod \"konnectivity-agent-wvvwk\" (UID: \"535498bf-bd89-46dd-9937-2f3adce9941b\") " pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790452 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-socket-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790509 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-var-lib-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-registration-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790561 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-etc-selinux\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.792491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790599 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysctl-d\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790624 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-k8s-cni-cncf-io\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-conf-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790676 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-systemd\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-var-lib-kubelet\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790720 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6684a217-290d-489a-8514-9b48a7940479-serviceca\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-systemd\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790770 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-cnibin\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790781 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-var-lib-openvswitch\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790797 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5plr\" (UniqueName: \"kubernetes.io/projected/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-kube-api-access-t5plr\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790771 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysconfig\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790872 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-kubernetes\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790899 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8r6c\" (UniqueName: \"kubernetes.io/projected/f3f32190-7a1c-439a-a3b6-c6fa26c74542-kube-api-access-w8r6c\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cni-binary-copy\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.790996 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a395fe49-f597-4ebe-bf24-38af015944f6-multus-daemon-config\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791022 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lnnp\" (UniqueName: \"kubernetes.io/projected/6dd64963-deab-4e30-973a-d6cebc6e0209-kube-api-access-4lnnp\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.793340 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791031 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-registration-dir\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791046 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tcth\" (UniqueName: \"kubernetes.io/projected/6684a217-290d-489a-8514-9b48a7940479-kube-api-access-8tcth\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791087 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-cnibin\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791110 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-etc-selinux\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791125 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-host-run-k8s-cni-cncf-io\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791180 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a395fe49-f597-4ebe-bf24-38af015944f6-multus-conf-dir\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791221 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b59a8c49-adcb-4435-8667-41a2203d706c-run-systemd\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791646 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a395fe49-f597-4ebe-bf24-38af015944f6-multus-daemon-config\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.794132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.791655 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b0278ab3-1cd5-4503-8554-43f8bc4c8398-cni-binary-copy\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.795201 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.795091 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b59a8c49-adcb-4435-8667-41a2203d706c-ovn-node-metrics-cert\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.795314 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.795269 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 02:09:45.795314 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.795286 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 02:09:45.795314 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.795303 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qpgck for pod openshift-network-diagnostics/network-check-target-qwwxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:45.795447 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:45.795358 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck podName:eef7433f-78c6-47eb-aad1-33b5138e03a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:46.295343534 +0000 UTC m=+3.045765168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qpgck" (UniqueName: "kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck") pod "network-check-target-qwwxl" (UID: "eef7433f-78c6-47eb-aad1-33b5138e03a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:45.796057 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.796038 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/535498bf-bd89-46dd-9937-2f3adce9941b-agent-certs\") pod \"konnectivity-agent-wvvwk\" (UID: \"535498bf-bd89-46dd-9937-2f3adce9941b\") " pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:45.798215 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.798192 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tf96\" (UniqueName: \"kubernetes.io/projected/af9fae31-f4c3-4823-b486-19c0322111bb-kube-api-access-4tf96\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:45.798637 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.798617 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6f8d\" (UniqueName: \"kubernetes.io/projected/a395fe49-f597-4ebe-bf24-38af015944f6-kube-api-access-h6f8d\") pod \"multus-dzhkk\" (UID: \"a395fe49-f597-4ebe-bf24-38af015944f6\") " pod="openshift-multus/multus-dzhkk" Jul 07 02:09:45.801064 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.801022 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfq2h\" (UniqueName: \"kubernetes.io/projected/b0278ab3-1cd5-4503-8554-43f8bc4c8398-kube-api-access-gfq2h\") pod \"multus-additional-cni-plugins-hhh5n\" (UID: \"b0278ab3-1cd5-4503-8554-43f8bc4c8398\") " pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:45.802612 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.802590 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5plr\" (UniqueName: \"kubernetes.io/projected/86a7f42e-d9a4-4b6a-966e-02f842fae6ef-kube-api-access-t5plr\") pod \"aws-ebs-csi-driver-node-m7lkb\" (UID: \"86a7f42e-d9a4-4b6a-966e-02f842fae6ef\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:45.803399 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.803367 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx28x\" (UniqueName: \"kubernetes.io/projected/b59a8c49-adcb-4435-8667-41a2203d706c-kube-api-access-hx28x\") pod \"ovnkube-node-9c6j6\" (UID: \"b59a8c49-adcb-4435-8667-41a2203d706c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.817977 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.817926 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" event={"ID":"419e48f62642a0af69b25fe0fa0d0553","Type":"ContainerStarted","Data":"41e561a0b0726930c12fcc68f847c0774cfd7d9d8bfd8010095c24f064a15e80"} Jul 07 02:09:45.819192 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.819165 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" event={"ID":"1c119a6373627d321976d8403f321294","Type":"ContainerStarted","Data":"b09b129f2cc95a756c790a39b964289bfba12138dd49aca56f410a43322c56ff"} Jul 07 02:09:45.891865 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.891827 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-run\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.891885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1b244eb-e66a-4f31-b82d-3ba053af3335-tmp\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.891978 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-run\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.891994 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-modprobe-d\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6684a217-290d-489a-8514-9b48a7940479-host\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.892270 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysctl-conf\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892270 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3f32190-7a1c-439a-a3b6-c6fa26c74542-iptables-alerter-script\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.892270 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892106 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6684a217-290d-489a-8514-9b48a7940479-host\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.892270 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892248 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysctl-conf\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892276 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-sys\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892291 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-lib-modules\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892312 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24s58\" (UniqueName: \"kubernetes.io/projected/a1b244eb-e66a-4f31-b82d-3ba053af3335-kube-api-access-24s58\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-host\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3f32190-7a1c-439a-a3b6-c6fa26c74542-host-slash\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892368 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6dd64963-deab-4e30-973a-d6cebc6e0209-hosts-file\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dd64963-deab-4e30-973a-d6cebc6e0209-tmp-dir\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-tuned\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892426 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysctl-d\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-systemd\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-var-lib-kubelet\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892486 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-sys\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892496 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6684a217-290d-489a-8514-9b48a7940479-serviceca\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892533 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysconfig\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892553 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-host\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892563 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-kubernetes\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892589 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8r6c\" (UniqueName: \"kubernetes.io/projected/f3f32190-7a1c-439a-a3b6-c6fa26c74542-kube-api-access-w8r6c\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892590 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3f32190-7a1c-439a-a3b6-c6fa26c74542-host-slash\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892635 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6dd64963-deab-4e30-973a-d6cebc6e0209-hosts-file\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892645 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lnnp\" (UniqueName: \"kubernetes.io/projected/6dd64963-deab-4e30-973a-d6cebc6e0209-kube-api-access-4lnnp\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892657 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f3f32190-7a1c-439a-a3b6-c6fa26c74542-iptables-alerter-script\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892669 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tcth\" (UniqueName: \"kubernetes.io/projected/6684a217-290d-489a-8514-9b48a7940479-kube-api-access-8tcth\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892751 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-lib-modules\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892760 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-kubernetes\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892811 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysconfig\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.892877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892880 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-var-lib-kubelet\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.893608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892888 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-systemd\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.893608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.892919 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dd64963-deab-4e30-973a-d6cebc6e0209-tmp-dir\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.893608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.893015 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-sysctl-d\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.893608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.893052 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6684a217-290d-489a-8514-9b48a7940479-serviceca\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.893608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.893176 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-modprobe-d\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.894717 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.894688 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1b244eb-e66a-4f31-b82d-3ba053af3335-tmp\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.895917 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.895891 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a1b244eb-e66a-4f31-b82d-3ba053af3335-etc-tuned\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.902083 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.902006 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24s58\" (UniqueName: \"kubernetes.io/projected/a1b244eb-e66a-4f31-b82d-3ba053af3335-kube-api-access-24s58\") pod \"tuned-j87hz\" (UID: \"a1b244eb-e66a-4f31-b82d-3ba053af3335\") " pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:45.902386 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.902365 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8r6c\" (UniqueName: \"kubernetes.io/projected/f3f32190-7a1c-439a-a3b6-c6fa26c74542-kube-api-access-w8r6c\") pod \"iptables-alerter-jnndx\" (UID: \"f3f32190-7a1c-439a-a3b6-c6fa26c74542\") " pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:45.902787 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.902769 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tcth\" (UniqueName: \"kubernetes.io/projected/6684a217-290d-489a-8514-9b48a7940479-kube-api-access-8tcth\") pod \"node-ca-b4gg5\" (UID: \"6684a217-290d-489a-8514-9b48a7940479\") " pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:45.905006 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.904969 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lnnp\" (UniqueName: \"kubernetes.io/projected/6dd64963-deab-4e30-973a-d6cebc6e0209-kube-api-access-4lnnp\") pod \"node-resolver-dcf4t\" (UID: \"6dd64963-deab-4e30-973a-d6cebc6e0209\") " pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:45.981992 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.981966 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:09:45.988047 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:45.988026 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" Jul 07 02:09:46.000195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.000172 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b4gg5" Jul 07 02:09:46.006041 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.006016 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dzhkk" Jul 07 02:09:46.015203 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.015180 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" Jul 07 02:09:46.033144 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.033108 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:09:46.042228 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.042119 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j87hz" Jul 07 02:09:46.047248 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.047215 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dcf4t" Jul 07 02:09:46.049985 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.049967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnndx" Jul 07 02:09:46.295232 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.295139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:46.295387 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.295258 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:46.295387 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.295271 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:46.295387 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.295319 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:09:47.295299188 +0000 UTC m=+4.045720841 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:46.295387 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.295373 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:46.295586 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.295433 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret podName:b459b352-3f73-4a90-a434-440bcb843220 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:47.295416625 +0000 UTC m=+4.045838274 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret") pod "global-pull-secret-syncer-d75g5" (UID: "b459b352-3f73-4a90-a434-440bcb843220") : object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:46.396630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.396594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:46.396808 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.396773 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 02:09:46.396808 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.396796 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 02:09:46.396808 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.396808 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qpgck for pod openshift-network-diagnostics/network-check-target-qwwxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:46.396970 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.396875 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck podName:eef7433f-78c6-47eb-aad1-33b5138e03a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:47.396855257 +0000 UTC m=+4.147276909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpgck" (UniqueName: "kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck") pod "network-check-target-qwwxl" (UID: "eef7433f-78c6-47eb-aad1-33b5138e03a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:46.721112 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.721040 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 02:04:45 +0000 UTC" deadline="2028-03-31 01:37:05.629871696 +0000 UTC" Jul 07 02:09:46.721112 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.721109 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15191h27m18.908774764s" Jul 07 02:09:46.812823 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:46.812794 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:46.812995 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:46.812923 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:46.967006 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.966978 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd64963_deab_4e30_973a_d6cebc6e0209.slice/crio-b3700aa948139f0205fe4cd69093ebbf17d61418982b72fbb53e189395d19df9 WatchSource:0}: Error finding container b3700aa948139f0205fe4cd69093ebbf17d61418982b72fbb53e189395d19df9: Status 404 returned error can't find the container with id b3700aa948139f0205fe4cd69093ebbf17d61418982b72fbb53e189395d19df9 Jul 07 02:09:46.968669 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.968634 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59a8c49_adcb_4435_8667_41a2203d706c.slice/crio-593eb0d10cc8283f2ac2b26cb28c829f9dae158f99a80113311ac6c0d7524253 WatchSource:0}: Error finding container 593eb0d10cc8283f2ac2b26cb28c829f9dae158f99a80113311ac6c0d7524253: Status 404 returned error can't find the container with id 593eb0d10cc8283f2ac2b26cb28c829f9dae158f99a80113311ac6c0d7524253 Jul 07 02:09:46.972745 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.972728 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0278ab3_1cd5_4503_8554_43f8bc4c8398.slice/crio-6f104e3802b1e8ef9bcb59340f60271d1f873dd88002f462ea530ecd7fcb523a WatchSource:0}: Error finding container 6f104e3802b1e8ef9bcb59340f60271d1f873dd88002f462ea530ecd7fcb523a: Status 404 returned error can't find the container with id 6f104e3802b1e8ef9bcb59340f60271d1f873dd88002f462ea530ecd7fcb523a Jul 07 02:09:46.973880 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.973858 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86a7f42e_d9a4_4b6a_966e_02f842fae6ef.slice/crio-1b723726663551f1b7a253d836c28a6c16bc67de7e1ab01094622adae4efe2c5 WatchSource:0}: Error finding container 1b723726663551f1b7a253d836c28a6c16bc67de7e1ab01094622adae4efe2c5: Status 404 returned error can't find the container with id 1b723726663551f1b7a253d836c28a6c16bc67de7e1ab01094622adae4efe2c5 Jul 07 02:09:46.974292 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.974267 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda395fe49_f597_4ebe_bf24_38af015944f6.slice/crio-63150e0bff46cc9c973bed115242cf70bba8af0c27576e5dc734b1e246a4fffb WatchSource:0}: Error finding container 63150e0bff46cc9c973bed115242cf70bba8af0c27576e5dc734b1e246a4fffb: Status 404 returned error can't find the container with id 63150e0bff46cc9c973bed115242cf70bba8af0c27576e5dc734b1e246a4fffb Jul 07 02:09:46.975256 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.975227 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b244eb_e66a_4f31_b82d_3ba053af3335.slice/crio-41ca21ca3185ac366c77006d9bb867e95154f2e9ec5838ed2902dab198d4030f WatchSource:0}: Error finding container 41ca21ca3185ac366c77006d9bb867e95154f2e9ec5838ed2902dab198d4030f: Status 404 returned error can't find the container with id 41ca21ca3185ac366c77006d9bb867e95154f2e9ec5838ed2902dab198d4030f Jul 07 02:09:46.977463 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.977438 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f32190_7a1c_439a_a3b6_c6fa26c74542.slice/crio-ff52d613d2a1f02087eb6571be6dfab3a5439540df9087e7270d7e8d7576ba4c WatchSource:0}: Error finding container ff52d613d2a1f02087eb6571be6dfab3a5439540df9087e7270d7e8d7576ba4c: Status 404 returned error can't find the container with id ff52d613d2a1f02087eb6571be6dfab3a5439540df9087e7270d7e8d7576ba4c Jul 07 02:09:46.978301 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.977975 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6684a217_290d_489a_8514_9b48a7940479.slice/crio-6cf3aef967add831cdcc9eed591ca121411453df2616d8f06bdd7300cfd9c9ea WatchSource:0}: Error finding container 6cf3aef967add831cdcc9eed591ca121411453df2616d8f06bdd7300cfd9c9ea: Status 404 returned error can't find the container with id 6cf3aef967add831cdcc9eed591ca121411453df2616d8f06bdd7300cfd9c9ea Jul 07 02:09:46.980548 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:09:46.980426 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535498bf_bd89_46dd_9937_2f3adce9941b.slice/crio-8a80bdf6479cf25035556effd05ed6889409253bc445f1c0982a016b58b960dc WatchSource:0}: Error finding container 8a80bdf6479cf25035556effd05ed6889409253bc445f1c0982a016b58b960dc: Status 404 returned error can't find the container with id 8a80bdf6479cf25035556effd05ed6889409253bc445f1c0982a016b58b960dc Jul 07 02:09:47.303032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.302847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:47.303144 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.303055 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:47.303144 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.302994 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:47.303144 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.303139 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:47.303261 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.303166 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret podName:b459b352-3f73-4a90-a434-440bcb843220 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:49.30313245 +0000 UTC m=+6.053554082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret") pod "global-pull-secret-syncer-d75g5" (UID: "b459b352-3f73-4a90-a434-440bcb843220") : object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:47.303261 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.303196 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:09:49.303181303 +0000 UTC m=+6.053602937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:47.403705 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.403672 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:47.403874 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.403837 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 02:09:47.403874 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.403862 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 02:09:47.403874 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.403875 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qpgck for pod openshift-network-diagnostics/network-check-target-qwwxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:47.404000 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.403924 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck podName:eef7433f-78c6-47eb-aad1-33b5138e03a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:49.403910004 +0000 UTC m=+6.154331638 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpgck" (UniqueName: "kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck") pod "network-check-target-qwwxl" (UID: "eef7433f-78c6-47eb-aad1-33b5138e03a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:47.721774 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.721662 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 02:04:45 +0000 UTC" deadline="2028-03-28 11:55:38.370640628 +0000 UTC" Jul 07 02:09:47.721774 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.721716 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15129h45m50.648928496s" Jul 07 02:09:47.814992 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.814956 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:47.815185 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.815085 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:09:47.815492 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.815473 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:47.815585 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:47.815566 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:47.837480 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.837448 2559 generic.go:356] "Generic (PLEG): container finished" podID="1c119a6373627d321976d8403f321294" containerID="56ea9b31ac19554dbd527e046d80164c54e3441f6a334f75b1a3f8bc9cb10e94" exitCode=0 Jul 07 02:09:47.837626 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.837530 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" event={"ID":"1c119a6373627d321976d8403f321294","Type":"ContainerDied","Data":"56ea9b31ac19554dbd527e046d80164c54e3441f6a334f75b1a3f8bc9cb10e94"} Jul 07 02:09:47.852614 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.852556 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvvwk" event={"ID":"535498bf-bd89-46dd-9937-2f3adce9941b","Type":"ContainerStarted","Data":"8a80bdf6479cf25035556effd05ed6889409253bc445f1c0982a016b58b960dc"} Jul 07 02:09:47.860493 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.860463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j87hz" event={"ID":"a1b244eb-e66a-4f31-b82d-3ba053af3335","Type":"ContainerStarted","Data":"41ca21ca3185ac366c77006d9bb867e95154f2e9ec5838ed2902dab198d4030f"} Jul 07 02:09:47.870672 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.870644 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"593eb0d10cc8283f2ac2b26cb28c829f9dae158f99a80113311ac6c0d7524253"} Jul 07 02:09:47.886104 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.886075 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dcf4t" event={"ID":"6dd64963-deab-4e30-973a-d6cebc6e0209","Type":"ContainerStarted","Data":"b3700aa948139f0205fe4cd69093ebbf17d61418982b72fbb53e189395d19df9"} Jul 07 02:09:47.902323 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.902292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" event={"ID":"419e48f62642a0af69b25fe0fa0d0553","Type":"ContainerStarted","Data":"cd255f04cf8e432c986a0130070a20e5977d1515867b5f258e93ed9f46e8c980"} Jul 07 02:09:47.906899 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.906288 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b4gg5" event={"ID":"6684a217-290d-489a-8514-9b48a7940479","Type":"ContainerStarted","Data":"6cf3aef967add831cdcc9eed591ca121411453df2616d8f06bdd7300cfd9c9ea"} Jul 07 02:09:47.909466 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.909404 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnndx" event={"ID":"f3f32190-7a1c-439a-a3b6-c6fa26c74542","Type":"ContainerStarted","Data":"ff52d613d2a1f02087eb6571be6dfab3a5439540df9087e7270d7e8d7576ba4c"} Jul 07 02:09:47.920849 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.920799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dzhkk" event={"ID":"a395fe49-f597-4ebe-bf24-38af015944f6","Type":"ContainerStarted","Data":"63150e0bff46cc9c973bed115242cf70bba8af0c27576e5dc734b1e246a4fffb"} Jul 07 02:09:47.925443 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.925369 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" event={"ID":"86a7f42e-d9a4-4b6a-966e-02f842fae6ef","Type":"ContainerStarted","Data":"1b723726663551f1b7a253d836c28a6c16bc67de7e1ab01094622adae4efe2c5"} Jul 07 02:09:47.929721 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:47.929697 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerStarted","Data":"6f104e3802b1e8ef9bcb59340f60271d1f873dd88002f462ea530ecd7fcb523a"} Jul 07 02:09:48.813066 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:48.812982 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:48.813616 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:48.813138 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:48.943994 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:48.943919 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" event={"ID":"1c119a6373627d321976d8403f321294","Type":"ContainerStarted","Data":"4cc0464aa15f9b00e757b5f3c1e41b0a8602aa7b5a26fa815e26a4767214e483"} Jul 07 02:09:48.969535 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:48.969476 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-30.ec2.internal" podStartSLOduration=3.969451654 podStartE2EDuration="3.969451654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:47.917789527 +0000 UTC m=+4.668211183" watchObservedRunningTime="2026-07-07 02:09:48.969451654 +0000 UTC m=+5.719873308" Jul 07 02:09:49.321264 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:49.321227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:49.321446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:49.321284 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:49.321502 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.321458 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:49.321568 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.321520 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:09:53.321500624 +0000 UTC m=+10.071922268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:49.321944 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.321898 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:49.322047 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.321952 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret podName:b459b352-3f73-4a90-a434-440bcb843220 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:53.321938108 +0000 UTC m=+10.072359753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret") pod "global-pull-secret-syncer-d75g5" (UID: "b459b352-3f73-4a90-a434-440bcb843220") : object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:49.422439 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:49.422399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:49.422617 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.422574 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 02:09:49.422617 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.422595 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 02:09:49.422617 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.422608 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qpgck for pod openshift-network-diagnostics/network-check-target-qwwxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:49.422886 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.422668 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck podName:eef7433f-78c6-47eb-aad1-33b5138e03a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:09:53.422649014 +0000 UTC m=+10.173070663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpgck" (UniqueName: "kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck") pod "network-check-target-qwwxl" (UID: "eef7433f-78c6-47eb-aad1-33b5138e03a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:49.813040 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:49.813005 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:49.813248 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:49.813013 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:49.813599 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.813329 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:49.813599 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:49.813369 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:09:50.812123 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:50.812083 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:50.812298 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:50.812251 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:51.815050 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:51.814276 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:51.815050 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:51.814423 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:09:51.815050 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:51.814831 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:51.815050 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:51.814930 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:52.812855 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:52.812816 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:52.813078 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:52.812971 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:53.357298 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:53.357254 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:53.357820 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:53.357315 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:53.357820 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.357436 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:53.357820 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.357471 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:53.357820 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.357515 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret podName:b459b352-3f73-4a90-a434-440bcb843220 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:01.357494753 +0000 UTC m=+18.107916401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret") pod "global-pull-secret-syncer-d75g5" (UID: "b459b352-3f73-4a90-a434-440bcb843220") : object "kube-system"/"original-pull-secret" not registered Jul 07 02:09:53.357820 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.357538 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:10:01.357527421 +0000 UTC m=+18.107949056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:09:53.458364 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:53.457978 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:53.458364 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.458186 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 02:09:53.458364 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.458211 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 02:09:53.458364 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.458225 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qpgck for pod openshift-network-diagnostics/network-check-target-qwwxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:53.458364 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.458289 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck podName:eef7433f-78c6-47eb-aad1-33b5138e03a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:01.458270809 +0000 UTC m=+18.208692461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpgck" (UniqueName: "kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck") pod "network-check-target-qwwxl" (UID: "eef7433f-78c6-47eb-aad1-33b5138e03a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:09:53.813989 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:53.813956 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:53.814189 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.814076 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:53.814189 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:53.814126 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:53.814316 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:53.814253 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:09:54.812496 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:54.812460 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:54.812910 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:54.812586 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:55.813005 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:55.812968 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:55.813469 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:55.813123 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:09:55.813469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:55.813208 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:55.813469 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:55.813324 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:56.813085 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:56.812860 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:56.813085 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:56.813012 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:57.813164 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.812967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:57.813550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.813040 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:57.813550 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:57.813294 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:57.813550 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:57.813321 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:09:57.967492 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.967448 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dcf4t" event={"ID":"6dd64963-deab-4e30-973a-d6cebc6e0209","Type":"ContainerStarted","Data":"8b5114accdd3195e3075c9dcfe169dcaa3b0967028178df3f02499e901f1f117"} Jul 07 02:09:57.969245 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.969211 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b4gg5" event={"ID":"6684a217-290d-489a-8514-9b48a7940479","Type":"ContainerStarted","Data":"5d58824cca329be1ce23c5b806dd6960569effcb3c1ef07676bfd7effde3d8ca"} Jul 07 02:09:57.970812 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.970775 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" event={"ID":"86a7f42e-d9a4-4b6a-966e-02f842fae6ef","Type":"ContainerStarted","Data":"500136c4c4fcac1478189350710900d580488767d546f412fede7266e939472a"} Jul 07 02:09:57.972418 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.972392 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0278ab3-1cd5-4503-8554-43f8bc4c8398" containerID="587867928e65530a92ab6fee9f67821bb9364291d8e61550187491c54b014915" exitCode=0 Jul 07 02:09:57.972526 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.972469 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerDied","Data":"587867928e65530a92ab6fee9f67821bb9364291d8e61550187491c54b014915"} Jul 07 02:09:57.974401 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.974088 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wvvwk" event={"ID":"535498bf-bd89-46dd-9937-2f3adce9941b","Type":"ContainerStarted","Data":"490392509caba69a12b774f30e0d5af4cce8f45ed4abea708bbc2c7a93258147"} Jul 07 02:09:57.975768 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.975722 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j87hz" event={"ID":"a1b244eb-e66a-4f31-b82d-3ba053af3335","Type":"ContainerStarted","Data":"8449e4051b0357c0a2a85ecf1270e08855f69220e9a7f722f07e2daa831801f0"} Jul 07 02:09:57.986371 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.986330 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-30.ec2.internal" podStartSLOduration=12.986317539 podStartE2EDuration="12.986317539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:48.970176534 +0000 UTC m=+5.720598189" watchObservedRunningTime="2026-07-07 02:09:57.986317539 +0000 UTC m=+14.736739193" Jul 07 02:09:57.986780 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.986753 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dcf4t" podStartSLOduration=4.596809288 podStartE2EDuration="14.986743118s" totalImagesPullingTime="10.38993383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:57.986124679 +0000 UTC m=+14.736546332" watchObservedRunningTime="2026-07-07 02:09:57.986743118 +0000 UTC m=+14.737164770" Jul 07 02:09:57.999601 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:57.999565 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wvvwk" podStartSLOduration=4.623080025 podStartE2EDuration="14.999551399s" totalImagesPullingTime="10.376471374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:57.999070821 +0000 UTC m=+14.749492476" watchObservedRunningTime="2026-07-07 02:09:57.999551399 +0000 UTC m=+14.749973054" Jul 07 02:09:58.035784 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:58.035733 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j87hz" podStartSLOduration=4.637911914 podStartE2EDuration="15.035716216s" totalImagesPullingTime="10.397804302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:58.035462454 +0000 UTC m=+14.785884108" watchObservedRunningTime="2026-07-07 02:09:58.035716216 +0000 UTC m=+14.786137868" Jul 07 02:09:58.049677 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:58.049634 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-b4gg5" podStartSLOduration=4.549650253 podStartE2EDuration="15.049618095s" totalImagesPullingTime="10.499967842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:58.049488752 +0000 UTC m=+14.799910413" watchObservedRunningTime="2026-07-07 02:09:58.049618095 +0000 UTC m=+14.800039755" Jul 07 02:09:58.812962 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:58.812926 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:09:58.813130 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:58.813084 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:09:58.979731 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:58.979490 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnndx" event={"ID":"f3f32190-7a1c-439a-a3b6-c6fa26c74542","Type":"ContainerStarted","Data":"0ba1324f5c5ca6b884c0f23079ae3a244803f243bb6a2413095179c5991f43dc"} Jul 07 02:09:58.996055 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:58.996006 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jnndx" podStartSLOduration=5.6167093900000005 podStartE2EDuration="15.995990836s" totalImagesPullingTime="10.379281446s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:09:58.995623189 +0000 UTC m=+15.746044852" watchObservedRunningTime="2026-07-07 02:09:58.995990836 +0000 UTC m=+15.746412488" Jul 07 02:09:59.815670 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:59.815629 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:09:59.815873 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:09:59.815634 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:09:59.815873 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:59.815786 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:09:59.815873 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:09:59.815837 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:00.596054 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:00.596019 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:10:00.596710 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:00.596691 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:10:00.812972 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:00.812938 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:00.813204 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:00.813082 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:01.424906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:01.424862 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:01.425115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:01.424919 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:01.425115 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.425033 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 02:10:01.425115 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.425054 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:10:01.425248 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.425117 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret podName:b459b352-3f73-4a90-a434-440bcb843220 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:17.425101519 +0000 UTC m=+34.175523151 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret") pod "global-pull-secret-syncer-d75g5" (UID: "b459b352-3f73-4a90-a434-440bcb843220") : object "kube-system"/"original-pull-secret" not registered Jul 07 02:10:01.425248 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.425136 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:10:17.425128859 +0000 UTC m=+34.175550489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:10:01.525829 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:01.525791 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:01.525990 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.525937 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 02:10:01.525990 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.525959 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 02:10:01.525990 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.525974 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qpgck for pod openshift-network-diagnostics/network-check-target-qwwxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:10:01.526111 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.526038 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck podName:eef7433f-78c6-47eb-aad1-33b5138e03a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:17.526018168 +0000 UTC m=+34.276439800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpgck" (UniqueName: "kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck") pod "network-check-target-qwwxl" (UID: "eef7433f-78c6-47eb-aad1-33b5138e03a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:10:01.815540 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:01.815472 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:01.816106 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:01.815472 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:01.816106 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.815595 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:01.816106 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:01.815663 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:02.812433 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:02.812399 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:02.812622 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:02.812538 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:03.816463 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:03.816424 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:03.816998 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:03.816530 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:03.816998 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:03.816430 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:03.816998 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:03.816749 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:04.812376 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:04.812340 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:04.812549 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:04.812488 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:05.816325 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:05.816113 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:05.816783 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:05.816430 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:05.816783 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:05.816428 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:05.816783 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:05.816503 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:05.996490 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:05.996455 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dzhkk" event={"ID":"a395fe49-f597-4ebe-bf24-38af015944f6","Type":"ContainerStarted","Data":"de187571623313b4fffcbe59102e619715233fb63b8569f26415feb0f1d78f49"} Jul 07 02:10:05.999314 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:05.999288 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"33d21396d6496fda8d8008c890b0276da1f4b815c597e1e7f550d346cef37cb7"} Jul 07 02:10:05.999442 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:05.999370 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"421a240252fdb024eec080c897888d5246f3af1184371765bc28b15314055306"} Jul 07 02:10:05.999442 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:05.999419 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"e83673f9f78bbb24eca38989ef00a7a60fa8b449dd45a062eaa34bf5fbfad7ef"} Jul 07 02:10:05.999544 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:05.999465 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"8045fb12180eafd4c0e5bf106b28014ae31c5147c3cf450b760aa25a14253f37"} Jul 07 02:10:06.015654 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.015599 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dzhkk" podStartSLOduration=4.308334143 podStartE2EDuration="23.015580034s" totalImagesPullingTime="18.707245891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:06.015438558 +0000 UTC m=+22.765860211" watchObservedRunningTime="2026-07-07 02:10:06.015580034 +0000 UTC m=+22.766001689" Jul 07 02:10:06.650346 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.650191 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 02:10:06.746062 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.745902 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T02:10:06.65034087Z","UUID":"b10fd525-5a97-4bc3-9349-a55e4e67fd25","Handler":null,"Name":"","Endpoint":""} Jul 07 02:10:06.747452 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.747433 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 02:10:06.747568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.747469 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 02:10:06.812380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.812296 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:06.812544 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:06.812400 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:06.972224 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.972192 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:10:06.972643 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.972315 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 02:10:06.972782 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:06.972762 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wvvwk" Jul 07 02:10:07.003528 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:07.003489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" event={"ID":"86a7f42e-d9a4-4b6a-966e-02f842fae6ef","Type":"ContainerStarted","Data":"d4c899af1a314bc6b64d18b55e7a8303c325381fe52eafb267bc0f6ac7749c67"} Jul 07 02:10:07.006790 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:07.006761 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"02e79f720b5c4246bac49912375ea47219537b39069acc3d80173481d7f8b700"} Jul 07 02:10:07.006930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:07.006832 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"e49309d1d15676de71104980b6d005d56eb2c73ae36673f1a628a1ef412d165e"} Jul 07 02:10:07.815298 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:07.815268 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:07.815432 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:07.815378 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:07.815432 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:07.815404 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:07.815513 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:07.815467 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:08.812026 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:08.811998 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:08.812549 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:08.812112 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:09.012609 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:09.012575 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" event={"ID":"86a7f42e-d9a4-4b6a-966e-02f842fae6ef","Type":"ContainerStarted","Data":"42c03d374a8980eb87e63f10509773b3a049e5f0fe4d62aa5d0353bc8ee45b8f"} Jul 07 02:10:09.017226 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:09.017192 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"650a01ff2d2e2d265c9a39c04dfba70edb03ee63d173b35e449d12d5dc0e629e"} Jul 07 02:10:09.031062 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:09.031015 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m7lkb" podStartSLOduration=5.370601652 podStartE2EDuration="26.031001913s" totalImagesPullingTime="20.660400261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:09.030562561 +0000 UTC m=+25.780984214" watchObservedRunningTime="2026-07-07 02:10:09.031001913 +0000 UTC m=+25.781423567" Jul 07 02:10:09.812396 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:09.812357 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:09.812852 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:09.812491 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:09.812852 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:09.812544 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:09.812852 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:09.812663 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:10.812562 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:10.812531 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:10.813110 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:10.812641 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:11.025424 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:11.025259 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" event={"ID":"b59a8c49-adcb-4435-8667-41a2203d706c","Type":"ContainerStarted","Data":"37d4bccac06253579a1b07e5f8dbd6ee6f402b6cdc3690f23918d3fff4693fc4"} Jul 07 02:10:11.025532 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:11.025467 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:10:11.025532 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:11.025509 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:10:11.043084 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:11.043068 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:10:11.052174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:11.052108 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" podStartSLOduration=9.334918429 podStartE2EDuration="28.052088937s" totalImagesPullingTime="18.717170508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:11.052041478 +0000 UTC m=+27.802463151" watchObservedRunningTime="2026-07-07 02:10:11.052088937 +0000 UTC m=+27.802510588" Jul 07 02:10:11.817252 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:11.817224 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:11.817790 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:11.817312 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:11.817790 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:11.817337 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:11.817790 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:11.817382 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:12.028561 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.028532 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0278ab3-1cd5-4503-8554-43f8bc4c8398" containerID="f29d6d91e979dc25ef1f605a079a253d6100e640e05d9f159d92ab3898faf904" exitCode=0 Jul 07 02:10:12.028748 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.028610 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerDied","Data":"f29d6d91e979dc25ef1f605a079a253d6100e640e05d9f159d92ab3898faf904"} Jul 07 02:10:12.029019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.029002 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:10:12.043726 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.043694 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:10:12.679639 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.679603 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d75g5"] Jul 07 02:10:12.679832 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.679729 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:12.679906 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:12.679840 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:12.684577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.684543 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qwwxl"] Jul 07 02:10:12.684713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.684643 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:12.684786 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:12.684732 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:12.685345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.685313 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44l58"] Jul 07 02:10:12.685472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:12.685421 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:12.685538 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:12.685516 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:14.034055 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:14.034015 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0278ab3-1cd5-4503-8554-43f8bc4c8398" containerID="527dc8cbe04374881eb7bea32e4b513e7cc5e7343798ccf3719f41d773a0681c" exitCode=0 Jul 07 02:10:14.034473 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:14.034094 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerDied","Data":"527dc8cbe04374881eb7bea32e4b513e7cc5e7343798ccf3719f41d773a0681c"} Jul 07 02:10:14.812791 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:14.812748 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:14.812791 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:14.812768 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:14.813009 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:14.812773 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:14.813009 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:14.812891 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:14.813120 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:14.813032 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:14.813202 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:14.813139 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:16.041032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:16.041003 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0278ab3-1cd5-4503-8554-43f8bc4c8398" containerID="636b6880010ed4af0778abf47ce14baed90426c00817532ba527de30810ea3c0" exitCode=0 Jul 07 02:10:16.041558 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:16.041053 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerDied","Data":"636b6880010ed4af0778abf47ce14baed90426c00817532ba527de30810ea3c0"} Jul 07 02:10:16.812554 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:16.812514 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:16.812749 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:16.812557 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:16.812749 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:16.812557 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:16.812749 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:16.812643 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d75g5" podUID="b459b352-3f73-4a90-a434-440bcb843220" Jul 07 02:10:16.812896 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:16.812753 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:10:16.812896 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:16.812819 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qwwxl" podUID="eef7433f-78c6-47eb-aad1-33b5138e03a5" Jul 07 02:10:17.451391 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:17.451355 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:17.451825 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:17.451409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:17.451825 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.451512 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 02:10:17.451825 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.451516 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:10:17.451825 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.451583 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret podName:b459b352-3f73-4a90-a434-440bcb843220 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:49.451563976 +0000 UTC m=+66.201985610 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret") pod "global-pull-secret-syncer-d75g5" (UID: "b459b352-3f73-4a90-a434-440bcb843220") : object "kube-system"/"original-pull-secret" not registered Jul 07 02:10:17.451825 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.451627 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:10:49.451606474 +0000 UTC m=+66.202028123 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 02:10:17.552397 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:17.552362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:17.552540 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.552488 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 02:10:17.552540 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.552505 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 02:10:17.552540 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.552515 2559 projected.go:196] Error preparing data for projected volume kube-api-access-qpgck for pod openshift-network-diagnostics/network-check-target-qwwxl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:10:17.552680 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:17.552566 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck podName:eef7433f-78c6-47eb-aad1-33b5138e03a5 nodeName:}" failed. No retries permitted until 2026-07-07 02:10:49.552550651 +0000 UTC m=+66.302972282 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpgck" (UniqueName: "kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck") pod "network-check-target-qwwxl" (UID: "eef7433f-78c6-47eb-aad1-33b5138e03a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 02:10:18.602308 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.602280 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-30.ec2.internal" event="NodeReady" Jul 07 02:10:18.602762 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.602400 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 02:10:18.651165 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.651106 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2276z"] Jul 07 02:10:18.653608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.653574 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.654486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.654460 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6fc7h"] Jul 07 02:10:18.656149 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.656124 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 02:10:18.656149 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.656135 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 02:10:18.656348 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.656139 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tv6md\"" Jul 07 02:10:18.656522 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.656504 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:18.659841 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.659823 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 02:10:18.660082 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.660060 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 02:10:18.660082 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.660072 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 02:10:18.660255 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.660100 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gjz6q\"" Jul 07 02:10:18.666060 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.665263 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2276z"] Jul 07 02:10:18.668628 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.668532 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6fc7h"] Jul 07 02:10:18.761363 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.761334 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hphlh\" (UniqueName: \"kubernetes.io/projected/e37a17b2-5148-48ab-aa5b-36dcd2569c84-kube-api-access-hphlh\") pod \"ingress-canary-6fc7h\" (UID: \"e37a17b2-5148-48ab-aa5b-36dcd2569c84\") " pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:18.761551 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.761382 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-config-volume\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.761551 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.761409 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.761551 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.761485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgqsz\" (UniqueName: \"kubernetes.io/projected/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-kube-api-access-kgqsz\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.761686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.761593 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-tmp-dir\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.761686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.761635 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e37a17b2-5148-48ab-aa5b-36dcd2569c84-cert\") pod \"ingress-canary-6fc7h\" (UID: \"e37a17b2-5148-48ab-aa5b-36dcd2569c84\") " pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:18.812546 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.812510 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:18.812546 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.812548 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:18.812738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.812550 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:18.815268 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.815244 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 02:10:18.815268 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.815252 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 02:10:18.815509 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.815496 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q64zw\"" Jul 07 02:10:18.815556 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.815529 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 02:10:18.815600 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.815562 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g22vd\"" Jul 07 02:10:18.815791 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.815778 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 02:10:18.862148 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862080 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-tmp-dir\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.862148 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e37a17b2-5148-48ab-aa5b-36dcd2569c84-cert\") pod \"ingress-canary-6fc7h\" (UID: \"e37a17b2-5148-48ab-aa5b-36dcd2569c84\") " pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:18.862349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862181 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hphlh\" (UniqueName: \"kubernetes.io/projected/e37a17b2-5148-48ab-aa5b-36dcd2569c84-kube-api-access-hphlh\") pod \"ingress-canary-6fc7h\" (UID: \"e37a17b2-5148-48ab-aa5b-36dcd2569c84\") " pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:18.862349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-config-volume\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.862349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.862349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862271 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgqsz\" (UniqueName: \"kubernetes.io/projected/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-kube-api-access-kgqsz\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.862526 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:18.862367 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:10:18.862526 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:18.862437 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:10:19.362414984 +0000 UTC m=+36.112836620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:10:18.862526 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862485 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-tmp-dir\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.862854 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.862833 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-config-volume\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.866385 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.866368 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e37a17b2-5148-48ab-aa5b-36dcd2569c84-cert\") pod \"ingress-canary-6fc7h\" (UID: \"e37a17b2-5148-48ab-aa5b-36dcd2569c84\") " pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:18.871120 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.871096 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgqsz\" (UniqueName: \"kubernetes.io/projected/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-kube-api-access-kgqsz\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:18.871337 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.871317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hphlh\" (UniqueName: \"kubernetes.io/projected/e37a17b2-5148-48ab-aa5b-36dcd2569c84-kube-api-access-hphlh\") pod \"ingress-canary-6fc7h\" (UID: \"e37a17b2-5148-48ab-aa5b-36dcd2569c84\") " pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:18.973437 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:18.973402 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6fc7h" Jul 07 02:10:19.139271 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:19.138947 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6fc7h"] Jul 07 02:10:19.143506 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:10:19.143474 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37a17b2_5148_48ab_aa5b_36dcd2569c84.slice/crio-dcb05332056577713734c214ada3f49deb39eaebc600ea6ca986250fa6e87b29 WatchSource:0}: Error finding container dcb05332056577713734c214ada3f49deb39eaebc600ea6ca986250fa6e87b29: Status 404 returned error can't find the container with id dcb05332056577713734c214ada3f49deb39eaebc600ea6ca986250fa6e87b29 Jul 07 02:10:19.367570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:19.367540 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:19.367754 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:19.367698 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:10:19.367810 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:19.367767 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:10:20.367751655 +0000 UTC m=+37.118173290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:10:20.054491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:20.054454 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6fc7h" event={"ID":"e37a17b2-5148-48ab-aa5b-36dcd2569c84","Type":"ContainerStarted","Data":"dcb05332056577713734c214ada3f49deb39eaebc600ea6ca986250fa6e87b29"} Jul 07 02:10:20.376813 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:20.376723 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:20.376967 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:20.376912 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:10:20.377014 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:20.376991 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:10:22.376970386 +0000 UTC m=+39.127392025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:10:21.060550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:21.060518 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0278ab3-1cd5-4503-8554-43f8bc4c8398" containerID="8a54fa198326b92dcf3fef59f62f705c81b5395b078a5f1156615e1eb290abbe" exitCode=0 Jul 07 02:10:21.060550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:21.060558 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerDied","Data":"8a54fa198326b92dcf3fef59f62f705c81b5395b078a5f1156615e1eb290abbe"} Jul 07 02:10:22.065995 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:22.065964 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0278ab3-1cd5-4503-8554-43f8bc4c8398" containerID="e3d0651fe4db68418b0cf664cedee5d49bd383d22a3eed2b38dec3bdfe730abe" exitCode=0 Jul 07 02:10:22.066385 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:22.066003 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerDied","Data":"e3d0651fe4db68418b0cf664cedee5d49bd383d22a3eed2b38dec3bdfe730abe"} Jul 07 02:10:22.392879 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:22.392802 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:22.393005 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:22.392954 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:10:22.393047 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:22.393019 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:10:26.393003032 +0000 UTC m=+43.143424664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:10:23.072435 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:23.072400 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" event={"ID":"b0278ab3-1cd5-4503-8554-43f8bc4c8398","Type":"ContainerStarted","Data":"0fddf9ce989ae5d6ccac869bbcdcf0394bfaeae0d8b2d59f9106970e136df243"} Jul 07 02:10:24.075760 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:24.075565 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6fc7h" event={"ID":"e37a17b2-5148-48ab-aa5b-36dcd2569c84","Type":"ContainerStarted","Data":"b1a0969b518b2f5cf7bf36eebdda337be8a6da062a31188332fb0935e5591dfc"} Jul 07 02:10:24.090425 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:24.090382 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hhh5n" podStartSLOduration=10.815556326 podStartE2EDuration="41.090368134s" totalImagesPullingTime="30.274811808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:23.099495827 +0000 UTC m=+39.849917480" watchObservedRunningTime="2026-07-07 02:10:24.090368134 +0000 UTC m=+40.840789786" Jul 07 02:10:24.090698 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:24.090680 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6fc7h" podStartSLOduration=1.678587357 podStartE2EDuration="6.090674191s" totalImagesPullingTime="4.412086834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:24.089399063 +0000 UTC m=+40.839820718" watchObservedRunningTime="2026-07-07 02:10:24.090674191 +0000 UTC m=+40.841095844" Jul 07 02:10:26.422863 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:26.422821 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:26.423237 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:26.422971 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:10:26.423237 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:26.423032 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:10:34.42301819 +0000 UTC m=+51.173439820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:10:34.480421 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:34.480379 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:34.481042 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:34.480541 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:10:34.481042 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:34.480617 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:10:50.480600309 +0000 UTC m=+67.231021940 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:10:44.055701 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:44.055674 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9c6j6" Jul 07 02:10:49.486016 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.485980 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:49.486402 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.486027 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:10:49.488826 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.488805 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 02:10:49.488905 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.488845 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 02:10:49.496732 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:49.496717 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 02:10:49.496797 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:49.496786 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:11:53.496765186 +0000 UTC m=+130.247186818 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : secret "metrics-daemon-secret" not found Jul 07 02:10:49.500436 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.500406 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b459b352-3f73-4a90-a434-440bcb843220-original-pull-secret\") pod \"global-pull-secret-syncer-d75g5\" (UID: \"b459b352-3f73-4a90-a434-440bcb843220\") " pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:49.587270 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.587223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:49.589929 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.589912 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 02:10:49.600693 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.600674 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 02:10:49.610813 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.610787 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpgck\" (UniqueName: \"kubernetes.io/projected/eef7433f-78c6-47eb-aad1-33b5138e03a5-kube-api-access-qpgck\") pod \"network-check-target-qwwxl\" (UID: \"eef7433f-78c6-47eb-aad1-33b5138e03a5\") " pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:49.730378 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.730347 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d75g5" Jul 07 02:10:49.737064 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.737014 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g22vd\"" Jul 07 02:10:49.745561 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.745538 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:49.859982 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.859955 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d75g5"] Jul 07 02:10:49.863300 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:10:49.863277 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb459b352_3f73_4a90_a434_440bcb843220.slice/crio-c48000e71d9e64ad303d4647b067e243cfca24ffbe671077cf3af820477a09c6 WatchSource:0}: Error finding container c48000e71d9e64ad303d4647b067e243cfca24ffbe671077cf3af820477a09c6: Status 404 returned error can't find the container with id c48000e71d9e64ad303d4647b067e243cfca24ffbe671077cf3af820477a09c6 Jul 07 02:10:49.874576 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:49.874554 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qwwxl"] Jul 07 02:10:49.877525 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:10:49.877503 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef7433f_78c6_47eb_aad1_33b5138e03a5.slice/crio-5acefcc937ecc7b7b2dfa7c801f88811abc03e5b7a9ca830413310a03b120091 WatchSource:0}: Error finding container 5acefcc937ecc7b7b2dfa7c801f88811abc03e5b7a9ca830413310a03b120091: Status 404 returned error can't find the container with id 5acefcc937ecc7b7b2dfa7c801f88811abc03e5b7a9ca830413310a03b120091 Jul 07 02:10:50.139680 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:50.139598 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qwwxl" event={"ID":"eef7433f-78c6-47eb-aad1-33b5138e03a5","Type":"ContainerStarted","Data":"5acefcc937ecc7b7b2dfa7c801f88811abc03e5b7a9ca830413310a03b120091"} Jul 07 02:10:50.140642 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:50.140622 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d75g5" event={"ID":"b459b352-3f73-4a90-a434-440bcb843220","Type":"ContainerStarted","Data":"c48000e71d9e64ad303d4647b067e243cfca24ffbe671077cf3af820477a09c6"} Jul 07 02:10:50.494051 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:50.494000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:10:50.494543 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:50.494183 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:10:50.494543 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:10:50.494269 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:11:22.494247486 +0000 UTC m=+99.244669120 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:10:55.154208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:55.154171 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qwwxl" event={"ID":"eef7433f-78c6-47eb-aad1-33b5138e03a5","Type":"ContainerStarted","Data":"3f6bc02ee9238afd212eceb80077f4d2d9971ef646d44cc9387869dfef3fd4db"} Jul 07 02:10:55.154638 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:55.154267 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:10:55.155447 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:55.155427 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d75g5" event={"ID":"b459b352-3f73-4a90-a434-440bcb843220","Type":"ContainerStarted","Data":"75acbb77ffb6f3fedaa95b89f65412537e992101084eae8e22471927d62c3a82"} Jul 07 02:10:55.168653 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:55.168611 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qwwxl" podStartSLOduration=67.322456128 podStartE2EDuration="1m12.168598974s" totalImagesPullingTime="4.846142846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:55.168192638 +0000 UTC m=+71.918614289" watchObservedRunningTime="2026-07-07 02:10:55.168598974 +0000 UTC m=+71.919020605" Jul 07 02:10:55.181342 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:10:55.181306 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d75g5" podStartSLOduration=67.315820026 podStartE2EDuration="1m12.181291749s" totalImagesPullingTime="4.865471723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:10:55.180935665 +0000 UTC m=+71.931357319" watchObservedRunningTime="2026-07-07 02:10:55.181291749 +0000 UTC m=+71.931713396" Jul 07 02:11:22.522010 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:11:22.521917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:11:22.522418 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:11:22.522081 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:11:22.522418 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:11:22.522177 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:12:26.522137787 +0000 UTC m=+163.272559421 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:11:26.160552 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:11:26.160521 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qwwxl" Jul 07 02:11:53.534271 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:11:53.534147 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:11:53.534271 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:11:53.534266 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 02:11:53.534880 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:11:53.534352 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:13:55.534335819 +0000 UTC m=+252.284757450 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : secret "metrics-daemon-secret" not found Jul 07 02:12:21.666958 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:12:21.666910 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2276z" podUID="b685c185-7f9e-4e17-ac8c-1906d1cd44ae" Jul 07 02:12:21.821847 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:12:21.821812 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:12:22.359506 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:12:22.359479 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2276z" Jul 07 02:12:26.553914 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:12:26.553876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:12:26.554481 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:12:26.554038 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 02:12:26.554481 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:12:26.554120 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls podName:b685c185-7f9e-4e17-ac8c-1906d1cd44ae nodeName:}" failed. No retries permitted until 2026-07-07 02:14:28.554098441 +0000 UTC m=+285.304520090 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls") pod "dns-default-2276z" (UID: "b685c185-7f9e-4e17-ac8c-1906d1cd44ae") : secret "dns-default-metrics-tls" not found Jul 07 02:12:32.812556 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:12:32.812502 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:13:55.544472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:55.544422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:13:55.544959 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:55.544578 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 02:13:55.544959 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:55.544644 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs podName:af9fae31-f4c3-4823-b486-19c0322111bb nodeName:}" failed. No retries permitted until 2026-07-07 02:15:57.544627662 +0000 UTC m=+374.295049293 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs") pod "network-metrics-daemon-44l58" (UID: "af9fae31-f4c3-4823-b486-19c0322111bb") : secret "metrics-daemon-secret" not found Jul 07 02:13:57.931526 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.931490 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797"] Jul 07 02:13:57.933585 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.933563 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:57.934687 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.934664 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4"] Jul 07 02:13:57.936434 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.936415 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-78d788fd9-7ww7b"] Jul 07 02:13:57.936551 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.936522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" Jul 07 02:13:57.938033 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.938017 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:57.938972 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.938929 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 02:13:57.938972 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.938949 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 02:13:57.939135 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.938979 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 02:13:57.939135 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.939030 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nhx44\"" Jul 07 02:13:57.939135 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.938955 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 02:13:57.939405 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.939392 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 02:13:57.939588 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.939576 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 02:13:57.939941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.939927 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bzxc5\"" Jul 07 02:13:57.940131 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.940097 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 02:13:57.940297 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.940285 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 02:13:57.941010 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.940991 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 02:13:57.941117 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.941032 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jr598\"" Jul 07 02:13:57.941205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.941116 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 02:13:57.941298 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.941286 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 02:13:57.941383 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.941308 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 02:13:57.962346 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.962322 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4"] Jul 07 02:13:57.969352 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.969304 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797"] Jul 07 02:13:57.970003 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:57.969985 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78d788fd9-7ww7b"] Jul 07 02:13:58.030052 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.030023 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c"] Jul 07 02:13:58.032162 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.032136 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.034937 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.034917 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 02:13:58.035042 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.034915 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 02:13:58.035323 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.035310 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8r5n6\"" Jul 07 02:13:58.035830 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.035811 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 02:13:58.035914 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.035891 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 02:13:58.045979 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.045958 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c"] Jul 07 02:13:58.057385 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057365 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-stats-auth\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.057500 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cstfh\" (UniqueName: \"kubernetes.io/projected/b54d838d-740f-4e40-9996-08f122a41670-kube-api-access-cstfh\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.057562 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057509 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.057619 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057563 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbwj2\" (UniqueName: \"kubernetes.io/projected/a7ece919-26cf-403e-90a9-bd15239a5e87-kube-api-access-cbwj2\") pod \"volume-data-source-validator-5d456fdbcd-6xzd4\" (UID: \"a7ece919-26cf-403e-90a9-bd15239a5e87\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" Jul 07 02:13:58.057619 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057595 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vvjh\" (UniqueName: \"kubernetes.io/projected/5bea1683-234d-447f-87b4-fbdf9186358b-kube-api-access-9vvjh\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.057697 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057626 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.057697 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057668 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b54d838d-740f-4e40-9996-08f122a41670-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.057697 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057690 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-default-certificate\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.057782 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.057707 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.158636 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b54d838d-740f-4e40-9996-08f122a41670-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.158636 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158637 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-default-certificate\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.158908 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158658 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.158908 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-stats-auth\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.158908 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.158752 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 02:13:58.158908 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158787 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cstfh\" (UniqueName: \"kubernetes.io/projected/b54d838d-740f-4e40-9996-08f122a41670-kube-api-access-cstfh\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.158908 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.158823 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:13:58.658806136 +0000 UTC m=+255.409227766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : secret "router-metrics-certs-default" not found Jul 07 02:13:58.158908 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl8kx\" (UniqueName: \"kubernetes.io/projected/a3df569c-4759-44bc-82fa-143c559207a8-kube-api-access-pl8kx\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158943 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3df569c-4759-44bc-82fa-143c559207a8-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158959 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3df569c-4759-44bc-82fa-143c559207a8-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.158985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbwj2\" (UniqueName: \"kubernetes.io/projected/a7ece919-26cf-403e-90a9-bd15239a5e87-kube-api-access-cbwj2\") pod \"volume-data-source-validator-5d456fdbcd-6xzd4\" (UID: \"a7ece919-26cf-403e-90a9-bd15239a5e87\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.159008 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vvjh\" (UniqueName: \"kubernetes.io/projected/5bea1683-234d-447f-87b4-fbdf9186358b-kube-api-access-9vvjh\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.159034 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.159050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.159087 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls podName:b54d838d-740f-4e40-9996-08f122a41670 nodeName:}" failed. No retries permitted until 2026-07-07 02:13:58.659071662 +0000 UTC m=+255.409493293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z8797" (UID: "b54d838d-740f-4e40-9996-08f122a41670") : secret "cluster-monitoring-operator-tls" not found Jul 07 02:13:58.159263 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.159142 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:13:58.659126993 +0000 UTC m=+255.409548671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : configmap references non-existent config key: service-ca.crt Jul 07 02:13:58.159709 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.159437 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b54d838d-740f-4e40-9996-08f122a41670-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.161081 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.161059 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-stats-auth\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.161211 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.161108 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-default-certificate\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.171518 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.171495 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cstfh\" (UniqueName: \"kubernetes.io/projected/b54d838d-740f-4e40-9996-08f122a41670-kube-api-access-cstfh\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.171628 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.171492 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vvjh\" (UniqueName: \"kubernetes.io/projected/5bea1683-234d-447f-87b4-fbdf9186358b-kube-api-access-9vvjh\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.171841 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.171825 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbwj2\" (UniqueName: \"kubernetes.io/projected/a7ece919-26cf-403e-90a9-bd15239a5e87-kube-api-access-cbwj2\") pod \"volume-data-source-validator-5d456fdbcd-6xzd4\" (UID: \"a7ece919-26cf-403e-90a9-bd15239a5e87\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" Jul 07 02:13:58.249934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.249850 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" Jul 07 02:13:58.260110 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.260085 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl8kx\" (UniqueName: \"kubernetes.io/projected/a3df569c-4759-44bc-82fa-143c559207a8-kube-api-access-pl8kx\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.260233 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.260127 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3df569c-4759-44bc-82fa-143c559207a8-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.260233 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.260164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3df569c-4759-44bc-82fa-143c559207a8-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.260853 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.260793 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3df569c-4759-44bc-82fa-143c559207a8-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.262853 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.262826 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3df569c-4759-44bc-82fa-143c559207a8-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.267686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.267666 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl8kx\" (UniqueName: \"kubernetes.io/projected/a3df569c-4759-44bc-82fa-143c559207a8-kube-api-access-pl8kx\") pod \"kube-storage-version-migrator-operator-846bf668f7-chj6c\" (UID: \"a3df569c-4759-44bc-82fa-143c559207a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.340576 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.340534 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" Jul 07 02:13:58.370266 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.370237 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4"] Jul 07 02:13:58.372951 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:13:58.372923 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ece919_26cf_403e_90a9_bd15239a5e87.slice/crio-be13cad033da32b312d2f5f34b8726f7c9f6504bf8bf0d89910e9a05f06397a0 WatchSource:0}: Error finding container be13cad033da32b312d2f5f34b8726f7c9f6504bf8bf0d89910e9a05f06397a0: Status 404 returned error can't find the container with id be13cad033da32b312d2f5f34b8726f7c9f6504bf8bf0d89910e9a05f06397a0 Jul 07 02:13:58.468933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.468905 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c"] Jul 07 02:13:58.471791 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:13:58.471761 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3df569c_4759_44bc_82fa_143c559207a8.slice/crio-236a56bba9561e9d85688eed65dc3e51676c15f5fc4b0b647a3ea4476f77be06 WatchSource:0}: Error finding container 236a56bba9561e9d85688eed65dc3e51676c15f5fc4b0b647a3ea4476f77be06: Status 404 returned error can't find the container with id 236a56bba9561e9d85688eed65dc3e51676c15f5fc4b0b647a3ea4476f77be06 Jul 07 02:13:58.577041 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.576952 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" event={"ID":"a7ece919-26cf-403e-90a9-bd15239a5e87","Type":"ContainerStarted","Data":"be13cad033da32b312d2f5f34b8726f7c9f6504bf8bf0d89910e9a05f06397a0"} Jul 07 02:13:58.578026 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.578002 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" event={"ID":"a3df569c-4759-44bc-82fa-143c559207a8","Type":"ContainerStarted","Data":"236a56bba9561e9d85688eed65dc3e51676c15f5fc4b0b647a3ea4476f77be06"} Jul 07 02:13:58.663329 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.663292 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.663497 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.663338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:58.663497 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:58.663384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:58.663497 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.663455 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:13:59.663437366 +0000 UTC m=+256.413859000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : configmap references non-existent config key: service-ca.crt Jul 07 02:13:58.663497 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.663477 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 02:13:58.663676 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.663505 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 02:13:58.663676 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.663522 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls podName:b54d838d-740f-4e40-9996-08f122a41670 nodeName:}" failed. No retries permitted until 2026-07-07 02:13:59.663507372 +0000 UTC m=+256.413929005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z8797" (UID: "b54d838d-740f-4e40-9996-08f122a41670") : secret "cluster-monitoring-operator-tls" not found Jul 07 02:13:58.663676 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:58.663565 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:13:59.663546029 +0000 UTC m=+256.413967675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : secret "router-metrics-certs-default" not found Jul 07 02:13:59.673005 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:59.672970 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:59.673474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:59.673018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:13:59.673474 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:59.673143 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 02:13:59.673474 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:59.673198 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:14:01.673166509 +0000 UTC m=+258.423588153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : configmap references non-existent config key: service-ca.crt Jul 07 02:13:59.673474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:13:59.673271 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:13:59.673474 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:59.673337 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:14:01.673318673 +0000 UTC m=+258.423740309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : secret "router-metrics-certs-default" not found Jul 07 02:13:59.673474 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:59.673383 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 02:13:59.673474 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:13:59.673423 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls podName:b54d838d-740f-4e40-9996-08f122a41670 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:01.67341116 +0000 UTC m=+258.423832793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z8797" (UID: "b54d838d-740f-4e40-9996-08f122a41670") : secret "cluster-monitoring-operator-tls" not found Jul 07 02:14:00.586339 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:00.586305 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" event={"ID":"a7ece919-26cf-403e-90a9-bd15239a5e87","Type":"ContainerStarted","Data":"5bb199e1ad94743777ae4f2ec0d0c7fef62027f10e15f5b67809036a74e7624b"} Jul 07 02:14:00.601976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:00.601915 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-6xzd4" podStartSLOduration=2.286286984 podStartE2EDuration="3.601893257s" totalImagesPullingTime="1.315606273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:00.601487008 +0000 UTC m=+257.351908661" watchObservedRunningTime="2026-07-07 02:14:00.601893257 +0000 UTC m=+257.352314910" Jul 07 02:14:01.589991 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:01.589951 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" event={"ID":"a3df569c-4759-44bc-82fa-143c559207a8","Type":"ContainerStarted","Data":"19639f3ef2863cf90d6e228f3892ceaf788130daf22252ef7f31ee7809a26133"} Jul 07 02:14:01.605389 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:01.605335 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" podStartSLOduration=1.132962636 podStartE2EDuration="3.605300088s" totalImagesPullingTime="2.472337452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:01.604609666 +0000 UTC m=+258.355031319" watchObservedRunningTime="2026-07-07 02:14:01.605300088 +0000 UTC m=+258.355721742" Jul 07 02:14:01.690027 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:01.689990 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:14:01.690247 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:01.690065 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:01.690247 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:01.690093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:01.690247 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:01.690187 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 02:14:01.690396 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:01.690261 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls podName:b54d838d-740f-4e40-9996-08f122a41670 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:05.690242284 +0000 UTC m=+262.440663917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z8797" (UID: "b54d838d-740f-4e40-9996-08f122a41670") : secret "cluster-monitoring-operator-tls" not found Jul 07 02:14:01.690396 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:01.690283 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:14:05.690267168 +0000 UTC m=+262.440688803 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : configmap references non-existent config key: service-ca.crt Jul 07 02:14:01.690627 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:01.690602 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 02:14:01.690693 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:01.690667 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:14:05.69065211 +0000 UTC m=+262.441073744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : secret "router-metrics-certs-default" not found Jul 07 02:14:02.852720 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:02.852684 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp"] Jul 07 02:14:02.854913 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:02.854890 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" Jul 07 02:14:02.857654 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:02.857634 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 02:14:02.857834 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:02.857657 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 02:14:02.858622 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:02.858606 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vkbpq\"" Jul 07 02:14:02.867352 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:02.867328 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp"] Jul 07 02:14:03.000956 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.000921 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rvs\" (UniqueName: \"kubernetes.io/projected/6325b1d9-f4d3-480d-bdb1-e379b30d64ae-kube-api-access-d2rvs\") pod \"migrator-d57d58cc9-kscgp\" (UID: \"6325b1d9-f4d3-480d-bdb1-e379b30d64ae\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" Jul 07 02:14:03.101530 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.101498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rvs\" (UniqueName: \"kubernetes.io/projected/6325b1d9-f4d3-480d-bdb1-e379b30d64ae-kube-api-access-d2rvs\") pod \"migrator-d57d58cc9-kscgp\" (UID: \"6325b1d9-f4d3-480d-bdb1-e379b30d64ae\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" Jul 07 02:14:03.109292 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.109229 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rvs\" (UniqueName: \"kubernetes.io/projected/6325b1d9-f4d3-480d-bdb1-e379b30d64ae-kube-api-access-d2rvs\") pod \"migrator-d57d58cc9-kscgp\" (UID: \"6325b1d9-f4d3-480d-bdb1-e379b30d64ae\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" Jul 07 02:14:03.163838 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.163798 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" Jul 07 02:14:03.285818 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.285791 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-htqsl"] Jul 07 02:14:03.288381 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.288357 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp"] Jul 07 02:14:03.288495 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.288454 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.293296 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.292616 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cczp7\"" Jul 07 02:14:03.293296 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.292897 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 02:14:03.293296 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.293145 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 02:14:03.293636 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.293473 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 02:14:03.293636 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.293483 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 02:14:03.301394 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.301371 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-htqsl"] Jul 07 02:14:03.403671 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.403642 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-data-volume\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.403886 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.403676 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.403886 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.403709 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-crio-socket\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.403886 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.403816 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xzvd\" (UniqueName: \"kubernetes.io/projected/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-api-access-5xzvd\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.403886 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.403850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504200 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.504139 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-data-volume\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.504206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.504267 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-crio-socket\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.504324 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xzvd\" (UniqueName: \"kubernetes.io/projected/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-api-access-5xzvd\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504372 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:03.504332 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 02:14:03.504372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.504360 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504599 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:03.504399 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:04.004376757 +0000 UTC m=+260.754798389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : configmap "kube-rbac-proxy" not found Jul 07 02:14:03.504599 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.504408 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-crio-socket\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504599 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:03.504465 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 02:14:03.504599 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.504498 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-data-volume\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.504599 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:03.504524 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:04.004507821 +0000 UTC m=+260.754929469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : secret "insights-runtime-extractor-tls" not found Jul 07 02:14:03.513022 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.513001 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xzvd\" (UniqueName: \"kubernetes.io/projected/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-api-access-5xzvd\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:03.595586 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:03.595552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" event={"ID":"6325b1d9-f4d3-480d-bdb1-e379b30d64ae","Type":"ContainerStarted","Data":"75830480cc32c13be62deb405c02d2cd018539c7488882a60465553a62e0efd5"} Jul 07 02:14:04.007912 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:04.007876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:04.008404 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:04.007960 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:04.008404 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:04.008022 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 02:14:04.008404 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:04.008083 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 02:14:04.008404 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:04.008092 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:05.00807473 +0000 UTC m=+261.758496365 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : configmap "kube-rbac-proxy" not found Jul 07 02:14:04.008404 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:04.008137 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:05.008121236 +0000 UTC m=+261.758542871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : secret "insights-runtime-extractor-tls" not found Jul 07 02:14:05.015083 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.014986 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:05.015471 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.015092 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:05.015471 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.015115 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 02:14:05.015471 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.015210 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:07.015191819 +0000 UTC m=+263.765613466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : configmap "kube-rbac-proxy" not found Jul 07 02:14:05.015471 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.015230 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 02:14:05.015471 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.015264 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:07.015254119 +0000 UTC m=+263.765675752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : secret "insights-runtime-extractor-tls" not found Jul 07 02:14:05.603001 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.602970 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" event={"ID":"6325b1d9-f4d3-480d-bdb1-e379b30d64ae","Type":"ContainerStarted","Data":"cb67d05a233a90016de641d5814e55f1fe0e3c401668052a088bf9a011a9de06"} Jul 07 02:14:05.603148 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.603019 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" event={"ID":"6325b1d9-f4d3-480d-bdb1-e379b30d64ae","Type":"ContainerStarted","Data":"0cca2bd41ffbba6e6f7279f44a19f664baf4d079f49104397c80f6328b2d3560"} Jul 07 02:14:05.623041 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.622992 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kscgp" podStartSLOduration=2.263064806 podStartE2EDuration="3.622976144s" totalImagesPullingTime="1.359911338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:05.62132943 +0000 UTC m=+262.371751082" watchObservedRunningTime="2026-07-07 02:14:05.622976144 +0000 UTC m=+262.373397796" Jul 07 02:14:05.720948 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.720908 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:05.721121 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.720993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:14:05.721121 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.721025 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:05.721121 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.721051 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 02:14:05.721121 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.721117 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:14:13.721101069 +0000 UTC m=+270.471522704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : secret "router-metrics-certs-default" not found Jul 07 02:14:05.721308 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.721137 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 02:14:05.721308 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.721174 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:14:13.721139131 +0000 UTC m=+270.471560767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : configmap references non-existent config key: service-ca.crt Jul 07 02:14:05.721308 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:05.721214 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls podName:b54d838d-740f-4e40-9996-08f122a41670 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:13.721202169 +0000 UTC m=+270.471623806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z8797" (UID: "b54d838d-740f-4e40-9996-08f122a41670") : secret "cluster-monitoring-operator-tls" not found Jul 07 02:14:05.873638 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:05.873564 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dcf4t_6dd64963-deab-4e30-973a-d6cebc6e0209/dns-node-resolver/0.log" Jul 07 02:14:06.874504 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:06.874475 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b4gg5_6684a217-290d-489a-8514-9b48a7940479/node-ca/0.log" Jul 07 02:14:07.033321 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:07.033282 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:07.033498 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:07.033349 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:07.033498 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:07.033430 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 02:14:07.033498 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:07.033446 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 02:14:07.033498 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:07.033497 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:11.033478986 +0000 UTC m=+267.783900616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : secret "insights-runtime-extractor-tls" not found Jul 07 02:14:07.033634 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:07.033514 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:11.033507344 +0000 UTC m=+267.783928974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : configmap "kube-rbac-proxy" not found Jul 07 02:14:07.674148 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:07.674118 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6fc7h_e37a17b2-5148-48ab-aa5b-36dcd2569c84/serve-healthcheck-canary/0.log" Jul 07 02:14:08.296595 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:08.296525 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kscgp_6325b1d9-f4d3-480d-bdb1-e379b30d64ae/migrator/0.log" Jul 07 02:14:08.473967 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:08.473939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kscgp_6325b1d9-f4d3-480d-bdb1-e379b30d64ae/graceful-termination/0.log" Jul 07 02:14:08.675558 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:08.675528 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-chj6c_a3df569c-4759-44bc-82fa-143c559207a8/kube-storage-version-migrator-operator/0.log" Jul 07 02:14:11.067592 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:11.067557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:11.067983 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:11.067613 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:11.067983 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:11.067681 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 02:14:11.067983 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:11.067693 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 02:14:11.067983 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:11.067742 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:19.067728248 +0000 UTC m=+275.818149879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : configmap "kube-rbac-proxy" not found Jul 07 02:14:11.067983 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:11.067755 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls podName:2ad31900-bfd3-4b69-9598-7fdfbb902fa7 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:19.067749141 +0000 UTC m=+275.818170771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-htqsl" (UID: "2ad31900-bfd3-4b69-9598-7fdfbb902fa7") : secret "insights-runtime-extractor-tls" not found Jul 07 02:14:13.789524 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:13.789495 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:14:13.789897 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:13.789540 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:13.789897 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:13.789579 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:13.789897 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:13.789649 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 02:14:13.789897 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:13.789714 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls podName:b54d838d-740f-4e40-9996-08f122a41670 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:29.78969762 +0000 UTC m=+286.540119251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z8797" (UID: "b54d838d-740f-4e40-9996-08f122a41670") : secret "cluster-monitoring-operator-tls" not found Jul 07 02:14:13.789897 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:13.789730 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle podName:5bea1683-234d-447f-87b4-fbdf9186358b nodeName:}" failed. No retries permitted until 2026-07-07 02:14:29.789722714 +0000 UTC m=+286.540144345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle") pod "router-default-78d788fd9-7ww7b" (UID: "5bea1683-234d-447f-87b4-fbdf9186358b") : configmap references non-existent config key: service-ca.crt Jul 07 02:14:13.791889 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:13.791868 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bea1683-234d-447f-87b4-fbdf9186358b-metrics-certs\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:19.135296 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:19.135244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:19.135751 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:19.135347 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:19.135792 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:19.135754 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:19.137555 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:19.137535 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ad31900-bfd3-4b69-9598-7fdfbb902fa7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-htqsl\" (UID: \"2ad31900-bfd3-4b69-9598-7fdfbb902fa7\") " pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:19.206101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:19.206073 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-htqsl" Jul 07 02:14:19.330780 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:19.330747 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-htqsl"] Jul 07 02:14:19.333582 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:19.333551 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad31900_bfd3_4b69_9598_7fdfbb902fa7.slice/crio-decf3feb7a359f1bcde2bd967d2f7d771a10bc3a3ef228072fbc6861b7b9e5ec WatchSource:0}: Error finding container decf3feb7a359f1bcde2bd967d2f7d771a10bc3a3ef228072fbc6861b7b9e5ec: Status 404 returned error can't find the container with id decf3feb7a359f1bcde2bd967d2f7d771a10bc3a3ef228072fbc6861b7b9e5ec Jul 07 02:14:19.642205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:19.642174 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htqsl" event={"ID":"2ad31900-bfd3-4b69-9598-7fdfbb902fa7","Type":"ContainerStarted","Data":"decf3feb7a359f1bcde2bd967d2f7d771a10bc3a3ef228072fbc6861b7b9e5ec"} Jul 07 02:14:23.656763 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:23.656717 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htqsl" event={"ID":"2ad31900-bfd3-4b69-9598-7fdfbb902fa7","Type":"ContainerStarted","Data":"9015e55f40ef22982c34a81d1818d0a1eb55b94f30d32ad58782bce8dfd7ba7f"} Jul 07 02:14:24.662840 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:24.662801 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htqsl" event={"ID":"2ad31900-bfd3-4b69-9598-7fdfbb902fa7","Type":"ContainerStarted","Data":"7c2924093bed196894207906fcce34d6d892ee06dc44bb8167f5558355907eaa"} Jul 07 02:14:25.360319 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:25.360277 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2276z" podUID="b685c185-7f9e-4e17-ac8c-1906d1cd44ae" Jul 07 02:14:25.666559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:25.666524 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2276z" Jul 07 02:14:26.538631 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.538605 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-746df6866f-zcg6t"] Jul 07 02:14:26.540686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.540672 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.543131 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.543112 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 02:14:26.544191 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.544167 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 02:14:26.544276 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.544206 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2bvdv\"" Jul 07 02:14:26.544333 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.544207 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 02:14:26.548468 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.548447 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 02:14:26.553676 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.553658 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-746df6866f-zcg6t"] Jul 07 02:14:26.674433 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.674400 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-htqsl" event={"ID":"2ad31900-bfd3-4b69-9598-7fdfbb902fa7","Type":"ContainerStarted","Data":"aa8f781eb78e97758d1b272c050f8fab1e20bbd27235aeb4faeb4473d83b85c3"} Jul 07 02:14:26.696393 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.696345 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-htqsl" podStartSLOduration=16.742108129000002 podStartE2EDuration="23.696329507s" totalImagesPullingTime="6.954221378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:14:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:26.695718448 +0000 UTC m=+283.446140101" watchObservedRunningTime="2026-07-07 02:14:26.696329507 +0000 UTC m=+283.446751161" Jul 07 02:14:26.698952 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.698932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17780b36-bc96-4268-99e5-0b71b432a3f4-trusted-ca\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.699022 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.698976 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17780b36-bc96-4268-99e5-0b71b432a3f4-registry-certificates\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.699022 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.699004 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17780b36-bc96-4268-99e5-0b71b432a3f4-image-registry-private-configuration\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.699087 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.699025 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-bound-sa-token\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.699087 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.699046 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz4kk\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-kube-api-access-lz4kk\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.699177 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.699137 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17780b36-bc96-4268-99e5-0b71b432a3f4-installation-pull-secrets\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.699242 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.699226 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-registry-tls\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.699276 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.699262 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17780b36-bc96-4268-99e5-0b71b432a3f4-ca-trust-extracted\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800377 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800345 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17780b36-bc96-4268-99e5-0b71b432a3f4-installation-pull-secrets\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800548 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-registry-tls\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800548 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800438 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17780b36-bc96-4268-99e5-0b71b432a3f4-ca-trust-extracted\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800548 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17780b36-bc96-4268-99e5-0b71b432a3f4-trusted-ca\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800548 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800526 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17780b36-bc96-4268-99e5-0b71b432a3f4-registry-certificates\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17780b36-bc96-4268-99e5-0b71b432a3f4-image-registry-private-configuration\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-bound-sa-token\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800700 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz4kk\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-kube-api-access-lz4kk\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.800921 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.800898 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17780b36-bc96-4268-99e5-0b71b432a3f4-ca-trust-extracted\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.801634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.801606 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17780b36-bc96-4268-99e5-0b71b432a3f4-registry-certificates\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.801791 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.801773 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17780b36-bc96-4268-99e5-0b71b432a3f4-trusted-ca\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.803016 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.802989 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17780b36-bc96-4268-99e5-0b71b432a3f4-installation-pull-secrets\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.803120 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.803044 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17780b36-bc96-4268-99e5-0b71b432a3f4-image-registry-private-configuration\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.803535 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.803518 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-registry-tls\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.810405 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.810385 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz4kk\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-kube-api-access-lz4kk\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.810733 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.810708 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17780b36-bc96-4268-99e5-0b71b432a3f4-bound-sa-token\") pod \"image-registry-746df6866f-zcg6t\" (UID: \"17780b36-bc96-4268-99e5-0b71b432a3f4\") " pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.851329 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.851292 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:26.970704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:26.970670 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-746df6866f-zcg6t"] Jul 07 02:14:26.973451 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:26.973424 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17780b36_bc96_4268_99e5_0b71b432a3f4.slice/crio-74d024c900ee27ab7d3f946f3baf094bca4605645292893442c7bf612abbfa7a WatchSource:0}: Error finding container 74d024c900ee27ab7d3f946f3baf094bca4605645292893442c7bf612abbfa7a: Status 404 returned error can't find the container with id 74d024c900ee27ab7d3f946f3baf094bca4605645292893442c7bf612abbfa7a Jul 07 02:14:27.679203 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:27.679145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-746df6866f-zcg6t" event={"ID":"17780b36-bc96-4268-99e5-0b71b432a3f4","Type":"ContainerStarted","Data":"8248b329691ab939cc717bf5a2d820457d66c679685369277f9e90a59b236e35"} Jul 07 02:14:27.679605 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:27.679217 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-746df6866f-zcg6t" event={"ID":"17780b36-bc96-4268-99e5-0b71b432a3f4","Type":"ContainerStarted","Data":"74d024c900ee27ab7d3f946f3baf094bca4605645292893442c7bf612abbfa7a"} Jul 07 02:14:27.699783 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:27.699735 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-746df6866f-zcg6t" podStartSLOduration=1.699718209 podStartE2EDuration="1.699718209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:27.698972738 +0000 UTC m=+284.449394391" watchObservedRunningTime="2026-07-07 02:14:27.699718209 +0000 UTC m=+284.450139861" Jul 07 02:14:28.615566 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:28.615529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:14:28.617856 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:28.617825 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b685c185-7f9e-4e17-ac8c-1906d1cd44ae-metrics-tls\") pod \"dns-default-2276z\" (UID: \"b685c185-7f9e-4e17-ac8c-1906d1cd44ae\") " pod="openshift-dns/dns-default-2276z" Jul 07 02:14:28.669442 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:28.669410 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tv6md\"" Jul 07 02:14:28.677539 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:28.677513 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2276z" Jul 07 02:14:28.682664 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:28.682641 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:28.797267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:28.797116 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2276z"] Jul 07 02:14:28.799745 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:28.799719 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb685c185_7f9e_4e17_ac8c_1906d1cd44ae.slice/crio-0130a47fb815b2d3a252f738e92813a48a42a624b634b7bf2afdedb08fd985a8 WatchSource:0}: Error finding container 0130a47fb815b2d3a252f738e92813a48a42a624b634b7bf2afdedb08fd985a8: Status 404 returned error can't find the container with id 0130a47fb815b2d3a252f738e92813a48a42a624b634b7bf2afdedb08fd985a8 Jul 07 02:14:29.687929 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:29.687886 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2276z" event={"ID":"b685c185-7f9e-4e17-ac8c-1906d1cd44ae","Type":"ContainerStarted","Data":"0130a47fb815b2d3a252f738e92813a48a42a624b634b7bf2afdedb08fd985a8"} Jul 07 02:14:29.826395 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:29.826360 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:14:29.826678 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:29.826453 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:29.827115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:29.827090 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bea1683-234d-447f-87b4-fbdf9186358b-service-ca-bundle\") pod \"router-default-78d788fd9-7ww7b\" (UID: \"5bea1683-234d-447f-87b4-fbdf9186358b\") " pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:29.829351 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:29.829330 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b54d838d-740f-4e40-9996-08f122a41670-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z8797\" (UID: \"b54d838d-740f-4e40-9996-08f122a41670\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:14:30.045092 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.045009 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" Jul 07 02:14:30.055224 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.055196 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:30.564526 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.564498 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797"] Jul 07 02:14:30.567259 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:30.567224 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54d838d_740f_4e40_9996_08f122a41670.slice/crio-23137baab96b294ae3142ca523f90151222078680986d40805150e889b4ccb36 WatchSource:0}: Error finding container 23137baab96b294ae3142ca523f90151222078680986d40805150e889b4ccb36: Status 404 returned error can't find the container with id 23137baab96b294ae3142ca523f90151222078680986d40805150e889b4ccb36 Jul 07 02:14:30.581142 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.581120 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78d788fd9-7ww7b"] Jul 07 02:14:30.583172 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:30.583136 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bea1683_234d_447f_87b4_fbdf9186358b.slice/crio-d5ac3b0e58ca1ac4a188d9fcf80fb64669133e80e8054bb39dc7ae94c5f78e95 WatchSource:0}: Error finding container d5ac3b0e58ca1ac4a188d9fcf80fb64669133e80e8054bb39dc7ae94c5f78e95: Status 404 returned error can't find the container with id d5ac3b0e58ca1ac4a188d9fcf80fb64669133e80e8054bb39dc7ae94c5f78e95 Jul 07 02:14:30.693419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.693326 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78d788fd9-7ww7b" event={"ID":"5bea1683-234d-447f-87b4-fbdf9186358b","Type":"ContainerStarted","Data":"9afeb1345b8941d9295aa45552848288f16d65da0f27518a28599983add2b3e3"} Jul 07 02:14:30.693419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.693401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78d788fd9-7ww7b" event={"ID":"5bea1683-234d-447f-87b4-fbdf9186358b","Type":"ContainerStarted","Data":"d5ac3b0e58ca1ac4a188d9fcf80fb64669133e80e8054bb39dc7ae94c5f78e95"} Jul 07 02:14:30.695008 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.694984 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2276z" event={"ID":"b685c185-7f9e-4e17-ac8c-1906d1cd44ae","Type":"ContainerStarted","Data":"932f2bc81b1d15540cdf18a112b351197a0e97d4858697c542a38e361566085b"} Jul 07 02:14:30.695110 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.695025 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2276z" event={"ID":"b685c185-7f9e-4e17-ac8c-1906d1cd44ae","Type":"ContainerStarted","Data":"83f3b0f9c4c31786e52edbabc21666e5b9c1e8c0b32ce73b2d8972d759b2ad26"} Jul 07 02:14:30.695110 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.695047 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2276z" Jul 07 02:14:30.696058 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.696031 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" event={"ID":"b54d838d-740f-4e40-9996-08f122a41670","Type":"ContainerStarted","Data":"23137baab96b294ae3142ca523f90151222078680986d40805150e889b4ccb36"} Jul 07 02:14:30.711095 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.711049 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-78d788fd9-7ww7b" podStartSLOduration=33.711034854 podStartE2EDuration="33.711034854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:30.710993598 +0000 UTC m=+287.461415253" watchObservedRunningTime="2026-07-07 02:14:30.711034854 +0000 UTC m=+287.461456512" Jul 07 02:14:30.730279 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:30.730223 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2276z" podStartSLOduration=251.066337162 podStartE2EDuration="4m12.730201865s" totalImagesPullingTime="1.663864703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:30.727692447 +0000 UTC m=+287.478114090" watchObservedRunningTime="2026-07-07 02:14:30.730201865 +0000 UTC m=+287.480623519" Jul 07 02:14:31.055757 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:31.055677 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:31.057951 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:31.057929 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:31.700698 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:31.700663 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:31.702114 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:31.702091 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-78d788fd9-7ww7b" Jul 07 02:14:33.172799 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.172761 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c"] Jul 07 02:14:33.175455 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.175427 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:33.177880 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.177851 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 02:14:33.178011 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.177900 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wfdxt\"" Jul 07 02:14:33.183219 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.183200 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c"] Jul 07 02:14:33.250623 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.250580 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7155e4ed-cdb5-4b92-889f-d5fe40fe0f18-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-wgr4c\" (UID: \"7155e4ed-cdb5-4b92-889f-d5fe40fe0f18\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:33.351453 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.351409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7155e4ed-cdb5-4b92-889f-d5fe40fe0f18-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-wgr4c\" (UID: \"7155e4ed-cdb5-4b92-889f-d5fe40fe0f18\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:33.351618 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:33.351538 2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 02:14:33.351618 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:33.351600 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7155e4ed-cdb5-4b92-889f-d5fe40fe0f18-tls-certificates podName:7155e4ed-cdb5-4b92-889f-d5fe40fe0f18 nodeName:}" failed. No retries permitted until 2026-07-07 02:14:33.851582373 +0000 UTC m=+290.602004004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/7155e4ed-cdb5-4b92-889f-d5fe40fe0f18-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-wgr4c" (UID: "7155e4ed-cdb5-4b92-889f-d5fe40fe0f18") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 02:14:33.708953 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.708917 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" event={"ID":"b54d838d-740f-4e40-9996-08f122a41670","Type":"ContainerStarted","Data":"433b8656d14c171500bd4fec22094cff74ea95babe2b25c29ce2cb2bc62957e3"} Jul 07 02:14:33.727018 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.726964 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z8797" podStartSLOduration=34.585515996 podStartE2EDuration="36.72694683s" totalImagesPullingTime="2.141430834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:13:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:33.725727707 +0000 UTC m=+290.476149359" watchObservedRunningTime="2026-07-07 02:14:33.72694683 +0000 UTC m=+290.477368482" Jul 07 02:14:33.855770 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.855733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7155e4ed-cdb5-4b92-889f-d5fe40fe0f18-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-wgr4c\" (UID: \"7155e4ed-cdb5-4b92-889f-d5fe40fe0f18\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:33.858299 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:33.858270 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7155e4ed-cdb5-4b92-889f-d5fe40fe0f18-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-wgr4c\" (UID: \"7155e4ed-cdb5-4b92-889f-d5fe40fe0f18\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:34.084842 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:34.084754 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:34.210666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:34.210646 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c"] Jul 07 02:14:34.212143 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:34.212118 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7155e4ed_cdb5_4b92_889f_d5fe40fe0f18.slice/crio-2e1a83549b9ebe12bec1827b8104bbb3d3ed5d8f54e4a4d0366865ecc0d11af0 WatchSource:0}: Error finding container 2e1a83549b9ebe12bec1827b8104bbb3d3ed5d8f54e4a4d0366865ecc0d11af0: Status 404 returned error can't find the container with id 2e1a83549b9ebe12bec1827b8104bbb3d3ed5d8f54e4a4d0366865ecc0d11af0 Jul 07 02:14:34.713870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:34.713829 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" event={"ID":"7155e4ed-cdb5-4b92-889f-d5fe40fe0f18","Type":"ContainerStarted","Data":"2e1a83549b9ebe12bec1827b8104bbb3d3ed5d8f54e4a4d0366865ecc0d11af0"} Jul 07 02:14:35.718726 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:35.718690 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" event={"ID":"7155e4ed-cdb5-4b92-889f-d5fe40fe0f18","Type":"ContainerStarted","Data":"1e8304e236f6094682072c2a56277bb2520d3a5062c92050be52f36954551a99"} Jul 07 02:14:35.719142 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:35.718886 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:35.723576 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:35.723556 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" Jul 07 02:14:35.734513 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:35.734467 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-wgr4c" podStartSLOduration=1.371435404 podStartE2EDuration="2.734455804s" totalImagesPullingTime="1.3630204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:35.733409049 +0000 UTC m=+292.483830703" watchObservedRunningTime="2026-07-07 02:14:35.734455804 +0000 UTC m=+292.484877457" Jul 07 02:14:35.813275 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:35.813239 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-44l58" podUID="af9fae31-f4c3-4823-b486-19c0322111bb" Jul 07 02:14:36.232424 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.232387 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv"] Jul 07 02:14:36.234757 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.234741 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.237980 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.237957 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zmvct\"" Jul 07 02:14:36.238116 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.238073 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 02:14:36.238116 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.238104 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 02:14:36.238252 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.238168 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 02:14:36.242668 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.242647 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv"] Jul 07 02:14:36.272882 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.272855 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nflm\" (UniqueName: \"kubernetes.io/projected/fa2668d1-3349-41b4-88e8-f5da2216074c-kube-api-access-9nflm\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.273031 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.272891 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa2668d1-3349-41b4-88e8-f5da2216074c-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.273031 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.272924 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.273031 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.272996 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.374035 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.373996 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.374210 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.374057 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nflm\" (UniqueName: \"kubernetes.io/projected/fa2668d1-3349-41b4-88e8-f5da2216074c-kube-api-access-9nflm\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.374210 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.374084 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa2668d1-3349-41b4-88e8-f5da2216074c-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.374210 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.374116 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.374210 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:36.374135 2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 02:14:36.374413 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:36.374223 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-tls podName:fa2668d1-3349-41b4-88e8-f5da2216074c nodeName:}" failed. No retries permitted until 2026-07-07 02:14:36.874206176 +0000 UTC m=+293.624627806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-p6zgv" (UID: "fa2668d1-3349-41b4-88e8-f5da2216074c") : secret "prometheus-operator-tls" not found Jul 07 02:14:36.374871 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.374843 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa2668d1-3349-41b4-88e8-f5da2216074c-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.376417 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.376396 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.382790 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.382764 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nflm\" (UniqueName: \"kubernetes.io/projected/fa2668d1-3349-41b4-88e8-f5da2216074c-kube-api-access-9nflm\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.876272 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.876241 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:36.878486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:36.878462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fa2668d1-3349-41b4-88e8-f5da2216074c-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-p6zgv\" (UID: \"fa2668d1-3349-41b4-88e8-f5da2216074c\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:37.143817 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:37.143736 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" Jul 07 02:14:37.271255 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:37.271230 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv"] Jul 07 02:14:37.273785 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:37.273757 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2668d1_3349_41b4_88e8_f5da2216074c.slice/crio-00946a82c6337a1dc872f4c94914f3eba7f4eee74181855ed971e5c1a0e914eb WatchSource:0}: Error finding container 00946a82c6337a1dc872f4c94914f3eba7f4eee74181855ed971e5c1a0e914eb: Status 404 returned error can't find the container with id 00946a82c6337a1dc872f4c94914f3eba7f4eee74181855ed971e5c1a0e914eb Jul 07 02:14:37.726482 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:37.726443 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" event={"ID":"fa2668d1-3349-41b4-88e8-f5da2216074c","Type":"ContainerStarted","Data":"00946a82c6337a1dc872f4c94914f3eba7f4eee74181855ed971e5c1a0e914eb"} Jul 07 02:14:39.734094 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:39.734058 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" event={"ID":"fa2668d1-3349-41b4-88e8-f5da2216074c","Type":"ContainerStarted","Data":"4c7690fa9601f396d475a977e96ad16d70676d9c9e2a0dbcad2cc1e255e5400d"} Jul 07 02:14:39.734455 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:39.734136 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" event={"ID":"fa2668d1-3349-41b4-88e8-f5da2216074c","Type":"ContainerStarted","Data":"4555963292b2a6c32b535999246de09421e0b25d241bf9f6d4e74eddb6549a96"} Jul 07 02:14:39.751415 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:39.751375 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-p6zgv" podStartSLOduration=2.26281347 podStartE2EDuration="3.751362441s" totalImagesPullingTime="1.488548971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:39.750453231 +0000 UTC m=+296.500874885" watchObservedRunningTime="2026-07-07 02:14:39.751362441 +0000 UTC m=+296.501784094" Jul 07 02:14:40.702553 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:40.702520 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2276z" Jul 07 02:14:41.569489 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.569289 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf"] Jul 07 02:14:41.572167 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.572130 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.574784 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.574762 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 02:14:41.574784 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.574773 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 02:14:41.574784 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.574768 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-frc5l\"" Jul 07 02:14:41.579318 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.579292 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54"] Jul 07 02:14:41.581395 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.581378 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.583965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.583946 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 02:14:41.584575 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.584553 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 02:14:41.584680 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.584581 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 02:14:41.584772 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.584751 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf"] Jul 07 02:14:41.584880 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.584824 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-87f52\"" Jul 07 02:14:41.597699 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.597675 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54"] Jul 07 02:14:41.598618 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.598601 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fjcl9"] Jul 07 02:14:41.601473 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.601456 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.604521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.604497 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 02:14:41.604620 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.604551 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m6bjf\"" Jul 07 02:14:41.604690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.604652 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 02:14:41.604690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.604667 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 02:14:41.613133 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613114 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pmsz\" (UniqueName: \"kubernetes.io/projected/05f26392-b740-44ae-9b89-2b619bf837c0-kube-api-access-9pmsz\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.613258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613142 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.613258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613183 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-tls\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613232 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613378 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613287 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s4pv\" (UniqueName: \"kubernetes.io/projected/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-kube-api-access-6s4pv\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613378 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613308 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f26392-b740-44ae-9b89-2b619bf837c0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.613378 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613324 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-root\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613378 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613346 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.613378 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613374 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-textfile\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.613589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb51ad16-a72d-49bc-a870-90a38a85c58e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.613589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613435 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-wtmp\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613456 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613474 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbzp5\" (UniqueName: \"kubernetes.io/projected/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-api-access-jbzp5\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.613589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613491 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-metrics-client-ca\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613507 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cb51ad16-a72d-49bc-a870-90a38a85c58e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.613834 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613589 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05f26392-b740-44ae-9b89-2b619bf837c0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.613834 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613627 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-sys\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.613834 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.613663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f26392-b740-44ae-9b89-2b619bf837c0-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.714263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714233 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05f26392-b740-44ae-9b89-2b619bf837c0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.714448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714268 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-sys\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.714448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f26392-b740-44ae-9b89-2b619bf837c0-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.714448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714424 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pmsz\" (UniqueName: \"kubernetes.io/projected/05f26392-b740-44ae-9b89-2b619bf837c0-kube-api-access-9pmsz\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.714448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714429 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-sys\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.714695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714456 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.714695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714499 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-tls\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.714695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714536 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.714695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714569 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s4pv\" (UniqueName: \"kubernetes.io/projected/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-kube-api-access-6s4pv\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.714695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714603 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f26392-b740-44ae-9b89-2b619bf837c0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.714695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714627 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-root\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.714695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714698 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-textfile\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714763 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb51ad16-a72d-49bc-a870-90a38a85c58e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714793 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-wtmp\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbzp5\" (UniqueName: \"kubernetes.io/projected/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-api-access-jbzp5\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714878 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-metrics-client-ca\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.715014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.714902 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cb51ad16-a72d-49bc-a870-90a38a85c58e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.715431 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715225 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05f26392-b740-44ae-9b89-2b619bf837c0-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.715431 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715310 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cb51ad16-a72d-49bc-a870-90a38a85c58e-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.715431 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715377 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.715680 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715655 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-textfile\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.715751 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715732 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-root\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.716035 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:41.715822 2559 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 07 02:14:41.716035 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715851 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.716035 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:14:41.715890 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-tls podName:cb51ad16-a72d-49bc-a870-90a38a85c58e nodeName:}" failed. No retries permitted until 2026-07-07 02:14:42.215872398 +0000 UTC m=+298.966294033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-clz54" (UID: "cb51ad16-a72d-49bc-a870-90a38a85c58e") : secret "kube-state-metrics-tls" not found Jul 07 02:14:41.716035 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715917 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb51ad16-a72d-49bc-a870-90a38a85c58e-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.716035 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.715991 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-wtmp\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.716391 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.716334 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-metrics-client-ca\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.717459 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.717421 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.717804 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.717777 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.717924 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.717872 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05f26392-b740-44ae-9b89-2b619bf837c0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.718022 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.718003 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-node-exporter-tls\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.718322 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.718303 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05f26392-b740-44ae-9b89-2b619bf837c0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.722955 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.722930 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s4pv\" (UniqueName: \"kubernetes.io/projected/5cb0b590-6421-4d6d-8594-678c9c5d8d9a-kube-api-access-6s4pv\") pod \"node-exporter-fjcl9\" (UID: \"5cb0b590-6421-4d6d-8594-678c9c5d8d9a\") " pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.723111 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.723097 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pmsz\" (UniqueName: \"kubernetes.io/projected/05f26392-b740-44ae-9b89-2b619bf837c0-kube-api-access-9pmsz\") pod \"openshift-state-metrics-54cf96dfcf-fwsvf\" (UID: \"05f26392-b740-44ae-9b89-2b619bf837c0\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.723532 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.723517 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbzp5\" (UniqueName: \"kubernetes.io/projected/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-api-access-jbzp5\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:41.881735 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.881661 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" Jul 07 02:14:41.910458 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:41.910418 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fjcl9" Jul 07 02:14:41.921820 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:41.921790 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb0b590_6421_4d6d_8594_678c9c5d8d9a.slice/crio-45ba8fa8ca0ff7c10f7108463b437256b07d3c93e9a262c99f17129beb9bd000 WatchSource:0}: Error finding container 45ba8fa8ca0ff7c10f7108463b437256b07d3c93e9a262c99f17129beb9bd000: Status 404 returned error can't find the container with id 45ba8fa8ca0ff7c10f7108463b437256b07d3c93e9a262c99f17129beb9bd000 Jul 07 02:14:42.015940 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.015830 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf"] Jul 07 02:14:42.018444 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:42.018420 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05f26392_b740_44ae_9b89_2b619bf837c0.slice/crio-63092c3fa03b0c88816a1483d3c58bfb4a4046dfc013a83d350ad5300336a9ee WatchSource:0}: Error finding container 63092c3fa03b0c88816a1483d3c58bfb4a4046dfc013a83d350ad5300336a9ee: Status 404 returned error can't find the container with id 63092c3fa03b0c88816a1483d3c58bfb4a4046dfc013a83d350ad5300336a9ee Jul 07 02:14:42.218841 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.218805 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:42.221307 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.221286 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb51ad16-a72d-49bc-a870-90a38a85c58e-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-clz54\" (UID: \"cb51ad16-a72d-49bc-a870-90a38a85c58e\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:42.490835 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.490750 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" Jul 07 02:14:42.647367 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.646769 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54"] Jul 07 02:14:42.652020 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:42.651984 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb51ad16_a72d_49bc_a870_90a38a85c58e.slice/crio-7b02220d5f0cc7b6c8e7f295905ff65cf6dd9702082037a24afbeaa84f8e31b4 WatchSource:0}: Error finding container 7b02220d5f0cc7b6c8e7f295905ff65cf6dd9702082037a24afbeaa84f8e31b4: Status 404 returned error can't find the container with id 7b02220d5f0cc7b6c8e7f295905ff65cf6dd9702082037a24afbeaa84f8e31b4 Jul 07 02:14:42.746772 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.746665 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjcl9" event={"ID":"5cb0b590-6421-4d6d-8594-678c9c5d8d9a","Type":"ContainerStarted","Data":"45ba8fa8ca0ff7c10f7108463b437256b07d3c93e9a262c99f17129beb9bd000"} Jul 07 02:14:42.748191 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.748141 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" event={"ID":"cb51ad16-a72d-49bc-a870-90a38a85c58e","Type":"ContainerStarted","Data":"7b02220d5f0cc7b6c8e7f295905ff65cf6dd9702082037a24afbeaa84f8e31b4"} Jul 07 02:14:42.750251 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.750222 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" event={"ID":"05f26392-b740-44ae-9b89-2b619bf837c0","Type":"ContainerStarted","Data":"6a4dbf1e53aa4f6bb622318a434d859fab5bbb67ebc94479c939660b2909e6f9"} Jul 07 02:14:42.750359 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.750281 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" event={"ID":"05f26392-b740-44ae-9b89-2b619bf837c0","Type":"ContainerStarted","Data":"fcd48818001944bbf594540611cd1888dedda7c708f5dba62042f900f35193f6"} Jul 07 02:14:42.750359 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:42.750309 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" event={"ID":"05f26392-b740-44ae-9b89-2b619bf837c0","Type":"ContainerStarted","Data":"63092c3fa03b0c88816a1483d3c58bfb4a4046dfc013a83d350ad5300336a9ee"} Jul 07 02:14:43.711827 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:43.711747 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 02:14:43.756556 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:43.756511 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" event={"ID":"05f26392-b740-44ae-9b89-2b619bf837c0","Type":"ContainerStarted","Data":"5a1f3b326b0800f66b7712e44bffce7badc318f3d54c9fdb6f355ba3cdd499fe"} Jul 07 02:14:43.758246 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:43.758219 2559 generic.go:356] "Generic (PLEG): container finished" podID="5cb0b590-6421-4d6d-8594-678c9c5d8d9a" containerID="4e0c5bff1a6a485f41d292930003c65c8778f1e9db53bcc93f3e267dadeeed13" exitCode=0 Jul 07 02:14:43.758368 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:43.758273 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjcl9" event={"ID":"5cb0b590-6421-4d6d-8594-678c9c5d8d9a","Type":"ContainerDied","Data":"4e0c5bff1a6a485f41d292930003c65c8778f1e9db53bcc93f3e267dadeeed13"} Jul 07 02:14:43.779682 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:43.779640 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fwsvf" podStartSLOduration=1.456219809 podStartE2EDuration="2.779623729s" totalImagesPullingTime="1.32340392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:43.778719097 +0000 UTC m=+300.529140761" watchObservedRunningTime="2026-07-07 02:14:43.779623729 +0000 UTC m=+300.530045381" Jul 07 02:14:44.772216 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:44.772185 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" event={"ID":"cb51ad16-a72d-49bc-a870-90a38a85c58e","Type":"ContainerStarted","Data":"a101224e44cfde35c79536f5d49071702134582b3d6a3576ea2d44e639e24e8c"} Jul 07 02:14:44.772705 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:44.772276 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" event={"ID":"cb51ad16-a72d-49bc-a870-90a38a85c58e","Type":"ContainerStarted","Data":"d6cb44bbc7b97dffef6535ed03b18b11ecd06f186d965e333ede6805f68ad968"} Jul 07 02:14:44.772705 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:44.772301 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" event={"ID":"cb51ad16-a72d-49bc-a870-90a38a85c58e","Type":"ContainerStarted","Data":"fc0d9941603ac60bc764e445c561a8b045e5d4d633dffc6b6c9c1e7f1000bbfe"} Jul 07 02:14:44.774458 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:44.774429 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjcl9" event={"ID":"5cb0b590-6421-4d6d-8594-678c9c5d8d9a","Type":"ContainerStarted","Data":"1b8f7b24a8654e8ebf5f32a475ed3cafc1ee674fabfbe4f628e850b9b555cbf3"} Jul 07 02:14:44.774585 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:44.774480 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjcl9" event={"ID":"5cb0b590-6421-4d6d-8594-678c9c5d8d9a","Type":"ContainerStarted","Data":"0087aaa53d09dc6d423b08497c2a70414413cf3381f8063da7b07d8c7bcb6345"} Jul 07 02:14:44.791572 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:44.791520 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-clz54" podStartSLOduration=2.163525236 podStartE2EDuration="3.791505976s" totalImagesPullingTime="1.62798074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:44.790938491 +0000 UTC m=+301.541360146" watchObservedRunningTime="2026-07-07 02:14:44.791505976 +0000 UTC m=+301.541927628" Jul 07 02:14:44.810552 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:44.810498 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fjcl9" podStartSLOduration=2.632671098 podStartE2EDuration="3.810463757s" totalImagesPullingTime="1.10071874s" totalInitContainerRuntime="77.073919ms" isStatefulPod=true podCreationTimestamp="2026-07-07 02:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:44.809377497 +0000 UTC m=+301.559799150" watchObservedRunningTime="2026-07-07 02:14:44.810463757 +0000 UTC m=+301.560885427" Jul 07 02:14:46.394781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.394747 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74d88bcbb-9zgtc"] Jul 07 02:14:46.398230 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.398209 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.401172 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.401135 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cnxzk\"" Jul 07 02:14:46.401285 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.401175 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 02:14:46.402111 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.402091 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 02:14:46.402304 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.402289 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 02:14:46.402395 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.402353 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 02:14:46.402456 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.402417 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 02:14:46.402456 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.402449 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 02:14:46.402665 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.402643 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 02:14:46.408191 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.408173 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 02:14:46.411579 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.411558 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d88bcbb-9zgtc"] Jul 07 02:14:46.455286 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.455253 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jhhl\" (UniqueName: \"kubernetes.io/projected/4ab631a2-4adf-4b5a-9a54-aa151a01826a-kube-api-access-9jhhl\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.455286 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.455287 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-oauth-serving-cert\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.455521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.455309 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-trusted-ca-bundle\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.455521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.455343 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-serving-cert\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.455521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.455423 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-service-ca\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.455521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.455461 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-oauth-config\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.455644 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.455550 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-config\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.556466 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.556427 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-config\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.556657 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.556478 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jhhl\" (UniqueName: \"kubernetes.io/projected/4ab631a2-4adf-4b5a-9a54-aa151a01826a-kube-api-access-9jhhl\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.556657 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.556502 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-oauth-serving-cert\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.556657 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.556548 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-trusted-ca-bundle\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.556657 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.556596 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-serving-cert\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.556657 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.556622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-service-ca\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.556657 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.556643 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-oauth-config\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.557304 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.557256 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-config\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.557411 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.557330 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-oauth-serving-cert\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.557476 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.557425 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-service-ca\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.557533 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.557469 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-trusted-ca-bundle\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.559093 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.559073 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-serving-cert\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.559205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.559189 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-oauth-config\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.563857 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.563832 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jhhl\" (UniqueName: \"kubernetes.io/projected/4ab631a2-4adf-4b5a-9a54-aa151a01826a-kube-api-access-9jhhl\") pod \"console-74d88bcbb-9zgtc\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.708319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.708288 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:14:46.778694 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.778663 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54f9dc9ffc-spl24"] Jul 07 02:14:46.783769 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.783575 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.786037 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.785993 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 02:14:46.786037 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.786008 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 02:14:46.786364 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.786145 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 02:14:46.786364 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.786355 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 02:14:46.786486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.786364 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 02:14:46.786524 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.786506 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mpfdv\"" Jul 07 02:14:46.792235 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.792215 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 02:14:46.794674 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.794653 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f9dc9ffc-spl24"] Jul 07 02:14:46.844991 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.844961 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d88bcbb-9zgtc"] Jul 07 02:14:46.848992 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:46.848964 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab631a2_4adf_4b5a_9a54_aa151a01826a.slice/crio-e1090febb357f6f3532ce4f80c4a00a8a658035b881f0050e74ab5d7b687af11 WatchSource:0}: Error finding container e1090febb357f6f3532ce4f80c4a00a8a658035b881f0050e74ab5d7b687af11: Status 404 returned error can't find the container with id e1090febb357f6f3532ce4f80c4a00a8a658035b881f0050e74ab5d7b687af11 Jul 07 02:14:46.850973 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.850956 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 02:14:46.860564 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860542 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-secret-telemeter-client\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.860647 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860575 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-serving-certs-ca-bundle\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.860647 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860599 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pplpq\" (UniqueName: \"kubernetes.io/projected/1ae94cd8-aecf-4414-86c5-27730557185a-kube-api-access-pplpq\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.860647 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860618 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.860768 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860668 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-federate-client-tls\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.860768 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-telemeter-client-tls\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.860768 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.860871 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.860777 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-metrics-client-ca\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.961522 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.961431 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-secret-telemeter-client\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.961522 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.961475 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-serving-certs-ca-bundle\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.961705 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.961636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pplpq\" (UniqueName: \"kubernetes.io/projected/1ae94cd8-aecf-4414-86c5-27730557185a-kube-api-access-pplpq\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.961705 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.961675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.961804 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.961718 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-federate-client-tls\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.961804 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.961749 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-telemeter-client-tls\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.961913 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.961835 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.962746 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.962362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-metrics-client-ca\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.962746 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.962711 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-serving-certs-ca-bundle\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.963999 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.963968 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-metrics-client-ca\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.964630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.964606 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae94cd8-aecf-4414-86c5-27730557185a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.964908 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.964883 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-secret-telemeter-client\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.967904 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.965341 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.967904 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.965559 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-telemeter-client-tls\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.967904 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.966651 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/1ae94cd8-aecf-4414-86c5-27730557185a-federate-client-tls\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:46.970244 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:46.970226 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pplpq\" (UniqueName: \"kubernetes.io/projected/1ae94cd8-aecf-4414-86c5-27730557185a-kube-api-access-pplpq\") pod \"telemeter-client-54f9dc9ffc-spl24\" (UID: \"1ae94cd8-aecf-4414-86c5-27730557185a\") " pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:47.096316 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:47.096282 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" Jul 07 02:14:47.224992 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:47.224918 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f9dc9ffc-spl24"] Jul 07 02:14:47.228245 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:14:47.228218 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae94cd8_aecf_4414_86c5_27730557185a.slice/crio-72dc77c1e9e7018044efc41038b50a47a8f6c7a1557146599b0c3fcdc6de6f11 WatchSource:0}: Error finding container 72dc77c1e9e7018044efc41038b50a47a8f6c7a1557146599b0c3fcdc6de6f11: Status 404 returned error can't find the container with id 72dc77c1e9e7018044efc41038b50a47a8f6c7a1557146599b0c3fcdc6de6f11 Jul 07 02:14:47.790351 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:47.790259 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" event={"ID":"1ae94cd8-aecf-4414-86c5-27730557185a","Type":"ContainerStarted","Data":"72dc77c1e9e7018044efc41038b50a47a8f6c7a1557146599b0c3fcdc6de6f11"} Jul 07 02:14:47.792471 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:47.792438 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d88bcbb-9zgtc" event={"ID":"4ab631a2-4adf-4b5a-9a54-aa151a01826a","Type":"ContainerStarted","Data":"e1090febb357f6f3532ce4f80c4a00a8a658035b881f0050e74ab5d7b687af11"} Jul 07 02:14:49.693495 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:49.693463 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-746df6866f-zcg6t" Jul 07 02:14:50.807011 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:50.806974 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" event={"ID":"1ae94cd8-aecf-4414-86c5-27730557185a","Type":"ContainerStarted","Data":"9a27a8335d95a8bc51c7eca6410d3c3f1c05ca7b4efbebeb252f2d3f11fa101e"} Jul 07 02:14:50.808372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:50.808348 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d88bcbb-9zgtc" event={"ID":"4ab631a2-4adf-4b5a-9a54-aa151a01826a","Type":"ContainerStarted","Data":"ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1"} Jul 07 02:14:50.811975 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:50.811958 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:14:50.825831 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:50.825789 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74d88bcbb-9zgtc" podStartSLOduration=1.303123993 podStartE2EDuration="4.825773176s" totalImagesPullingTime="3.522649183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:50.824501607 +0000 UTC m=+307.574923260" watchObservedRunningTime="2026-07-07 02:14:50.825773176 +0000 UTC m=+307.576194830" Jul 07 02:14:51.816479 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:51.816395 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" event={"ID":"1ae94cd8-aecf-4414-86c5-27730557185a","Type":"ContainerStarted","Data":"d2f298eabd2310ba0c6707514fb98a52bc6f8a1e36da5c03eb27c0fd97e90e9c"} Jul 07 02:14:51.816479 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:51.816450 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" event={"ID":"1ae94cd8-aecf-4414-86c5-27730557185a","Type":"ContainerStarted","Data":"02e13fb7a6352905ba01535e886f7b4b59fa368d086ce88344fd3a63f67eca9e"} Jul 07 02:14:51.839032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:51.838983 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54f9dc9ffc-spl24" podStartSLOduration=1.724116669 podStartE2EDuration="5.838967521s" totalImagesPullingTime="4.114850852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:14:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:14:51.837253208 +0000 UTC m=+308.587674861" watchObservedRunningTime="2026-07-07 02:14:51.838967521 +0000 UTC m=+308.589389174" Jul 07 02:14:56.597986 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:56.597945 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d88bcbb-9zgtc"] Jul 07 02:14:56.709143 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:14:56.709105 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:15:11.895961 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:11.895924 2559 generic.go:356] "Generic (PLEG): container finished" podID="a3df569c-4759-44bc-82fa-143c559207a8" containerID="19639f3ef2863cf90d6e228f3892ceaf788130daf22252ef7f31ee7809a26133" exitCode=0 Jul 07 02:15:11.896441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:11.895970 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" event={"ID":"a3df569c-4759-44bc-82fa-143c559207a8","Type":"ContainerDied","Data":"19639f3ef2863cf90d6e228f3892ceaf788130daf22252ef7f31ee7809a26133"} Jul 07 02:15:11.896441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:11.896293 2559 scope.go:117] "RemoveContainer" containerID="19639f3ef2863cf90d6e228f3892ceaf788130daf22252ef7f31ee7809a26133" Jul 07 02:15:12.901102 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:12.901069 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-chj6c" event={"ID":"a3df569c-4759-44bc-82fa-143c559207a8","Type":"ContainerStarted","Data":"f7f2fede5d8c886f3ad94fa8aae4fb6356a7ab973b6feb6bf63ebf6850a1a3f9"} Jul 07 02:15:21.619598 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.619525 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74d88bcbb-9zgtc" podUID="4ab631a2-4adf-4b5a-9a54-aa151a01826a" containerName="console" containerID="cri-o://ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1" gracePeriod=15 Jul 07 02:15:21.896204 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.896115 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:15:21.933490 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.933455 2559 generic.go:356] "Generic (PLEG): container finished" podID="4ab631a2-4adf-4b5a-9a54-aa151a01826a" containerID="ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1" exitCode=0 Jul 07 02:15:21.933646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.933499 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d88bcbb-9zgtc" Jul 07 02:15:21.933646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.933506 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d88bcbb-9zgtc" event={"ID":"4ab631a2-4adf-4b5a-9a54-aa151a01826a","Type":"ContainerDied","Data":"ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1"} Jul 07 02:15:21.933646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.933543 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d88bcbb-9zgtc" event={"ID":"4ab631a2-4adf-4b5a-9a54-aa151a01826a","Type":"ContainerDied","Data":"e1090febb357f6f3532ce4f80c4a00a8a658035b881f0050e74ab5d7b687af11"} Jul 07 02:15:21.933646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.933564 2559 scope.go:117] "RemoveContainer" containerID="ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1" Jul 07 02:15:21.957964 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.957941 2559 scope.go:117] "RemoveContainer" containerID="ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1" Jul 07 02:15:21.958304 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:15:21.958277 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1\": container with ID starting with ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1 not found: ID does not exist" containerID="ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1" Jul 07 02:15:21.958402 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:21.958313 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1"} err="failed to get container status \"ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1\": rpc error: code = NotFound desc = could not find container \"ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1\": container with ID starting with ec56fda2b82a06f25989614ae66a1a219dacbba78d7ed54a3ed71e2ef7a9d5d1 not found: ID does not exist" Jul 07 02:15:22.068745 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.068711 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-oauth-config\") pod \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " Jul 07 02:15:22.068941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.068788 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-serving-cert\") pod \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " Jul 07 02:15:22.068941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.068814 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-config\") pod \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " Jul 07 02:15:22.068941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.068837 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-oauth-serving-cert\") pod \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " Jul 07 02:15:22.068941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.068868 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jhhl\" (UniqueName: \"kubernetes.io/projected/4ab631a2-4adf-4b5a-9a54-aa151a01826a-kube-api-access-9jhhl\") pod \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " Jul 07 02:15:22.068941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.068910 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-trusted-ca-bundle\") pod \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " Jul 07 02:15:22.068941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.068935 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-service-ca\") pod \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\" (UID: \"4ab631a2-4adf-4b5a-9a54-aa151a01826a\") " Jul 07 02:15:22.069308 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.069279 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4ab631a2-4adf-4b5a-9a54-aa151a01826a" (UID: "4ab631a2-4adf-4b5a-9a54-aa151a01826a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:15:22.069428 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.069402 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-config" (OuterVolumeSpecName: "console-config") pod "4ab631a2-4adf-4b5a-9a54-aa151a01826a" (UID: "4ab631a2-4adf-4b5a-9a54-aa151a01826a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:15:22.069494 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.069402 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4ab631a2-4adf-4b5a-9a54-aa151a01826a" (UID: "4ab631a2-4adf-4b5a-9a54-aa151a01826a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:15:22.069494 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.069454 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-service-ca" (OuterVolumeSpecName: "service-ca") pod "4ab631a2-4adf-4b5a-9a54-aa151a01826a" (UID: "4ab631a2-4adf-4b5a-9a54-aa151a01826a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:15:22.071087 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.071060 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4ab631a2-4adf-4b5a-9a54-aa151a01826a" (UID: "4ab631a2-4adf-4b5a-9a54-aa151a01826a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 02:15:22.071331 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.071304 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4ab631a2-4adf-4b5a-9a54-aa151a01826a" (UID: "4ab631a2-4adf-4b5a-9a54-aa151a01826a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 02:15:22.071418 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.071343 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab631a2-4adf-4b5a-9a54-aa151a01826a-kube-api-access-9jhhl" (OuterVolumeSpecName: "kube-api-access-9jhhl") pod "4ab631a2-4adf-4b5a-9a54-aa151a01826a" (UID: "4ab631a2-4adf-4b5a-9a54-aa151a01826a"). InnerVolumeSpecName "kube-api-access-9jhhl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:15:22.170513 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.170485 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-oauth-config\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:15:22.170513 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.170511 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-serving-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:15:22.170690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.170520 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-console-config\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:15:22.170690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.170529 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-oauth-serving-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:15:22.170690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.170538 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jhhl\" (UniqueName: \"kubernetes.io/projected/4ab631a2-4adf-4b5a-9a54-aa151a01826a-kube-api-access-9jhhl\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:15:22.170690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.170547 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-trusted-ca-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:15:22.170690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.170555 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4ab631a2-4adf-4b5a-9a54-aa151a01826a-service-ca\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:15:22.255601 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.255562 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d88bcbb-9zgtc"] Jul 07 02:15:22.259510 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:22.259480 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74d88bcbb-9zgtc"] Jul 07 02:15:23.816330 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:23.816296 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab631a2-4adf-4b5a-9a54-aa151a01826a" path="/var/lib/kubelet/pods/4ab631a2-4adf-4b5a-9a54-aa151a01826a/volumes" Jul 07 02:15:28.665900 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:28.665874 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78d788fd9-7ww7b_5bea1683-234d-447f-87b4-fbdf9186358b/router/0.log" Jul 07 02:15:28.678348 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:28.678327 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6fc7h_e37a17b2-5148-48ab-aa5b-36dcd2569c84/serve-healthcheck-canary/0.log" Jul 07 02:15:57.551264 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:57.551222 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:15:57.553629 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:57.553603 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/af9fae31-f4c3-4823-b486-19c0322111bb-metrics-certs\") pod \"network-metrics-daemon-44l58\" (UID: \"af9fae31-f4c3-4823-b486-19c0322111bb\") " pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:15:57.715295 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:57.715256 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q64zw\"" Jul 07 02:15:57.723607 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:57.723583 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-44l58" Jul 07 02:15:57.858731 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:57.858707 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-44l58"] Jul 07 02:15:57.861339 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:15:57.861304 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9fae31_f4c3_4823_b486_19c0322111bb.slice/crio-e03abb4945df70aa8f48ffe4d6d6a9595ab4e68c853fa15a69797d3f9f54251f WatchSource:0}: Error finding container e03abb4945df70aa8f48ffe4d6d6a9595ab4e68c853fa15a69797d3f9f54251f: Status 404 returned error can't find the container with id e03abb4945df70aa8f48ffe4d6d6a9595ab4e68c853fa15a69797d3f9f54251f Jul 07 02:15:58.068037 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:58.067946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44l58" event={"ID":"af9fae31-f4c3-4823-b486-19c0322111bb","Type":"ContainerStarted","Data":"e03abb4945df70aa8f48ffe4d6d6a9595ab4e68c853fa15a69797d3f9f54251f"} Jul 07 02:15:59.073291 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:59.073204 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44l58" event={"ID":"af9fae31-f4c3-4823-b486-19c0322111bb","Type":"ContainerStarted","Data":"324bbbb3198eac9473308ea4a1026cde6ed5de4ef55d4a07371ae46c46a7f99f"} Jul 07 02:15:59.073291 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:59.073266 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-44l58" event={"ID":"af9fae31-f4c3-4823-b486-19c0322111bb","Type":"ContainerStarted","Data":"c7a1a0401a4db59e73689f3af6246ad5fa0fd25660c3cd69f4df6d4984a18700"} Jul 07 02:15:59.091589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:15:59.091540 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-44l58" podStartSLOduration=375.180518554 podStartE2EDuration="6m16.091521352s" totalImagesPullingTime="911.002798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:15:59.09130783 +0000 UTC m=+375.841729483" watchObservedRunningTime="2026-07-07 02:15:59.091521352 +0000 UTC m=+375.841943005" Jul 07 02:16:00.846377 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.846345 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9fcbf6949-x7q8x"] Jul 07 02:16:00.846738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.846621 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab631a2-4adf-4b5a-9a54-aa151a01826a" containerName="console" Jul 07 02:16:00.846738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.846631 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab631a2-4adf-4b5a-9a54-aa151a01826a" containerName="console" Jul 07 02:16:00.846738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.846706 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab631a2-4adf-4b5a-9a54-aa151a01826a" containerName="console" Jul 07 02:16:00.848770 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.848745 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:00.851148 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.851096 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 02:16:00.851651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.851626 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 02:16:00.851808 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.851771 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 02:16:00.851930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.851850 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 02:16:00.852090 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.852017 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 02:16:00.852438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.852418 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cnxzk\"" Jul 07 02:16:00.852951 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.852928 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 02:16:00.853052 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.852927 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 02:16:00.861566 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.861543 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 02:16:00.864383 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.864361 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9fcbf6949-x7q8x"] Jul 07 02:16:00.978793 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.978756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-serving-cert\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:00.978793 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.978798 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8tq7\" (UniqueName: \"kubernetes.io/projected/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-kube-api-access-q8tq7\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:00.978995 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.978825 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-config\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:00.978995 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.978858 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-oauth-config\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:00.978995 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.978899 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-service-ca\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:00.978995 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.978913 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-trusted-ca-bundle\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:00.978995 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:00.978929 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-oauth-serving-cert\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.079282 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.079243 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-serving-cert\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.079282 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.079281 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8tq7\" (UniqueName: \"kubernetes.io/projected/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-kube-api-access-q8tq7\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.079547 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.079304 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-config\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.079547 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.079324 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-oauth-config\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.079547 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.079352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-service-ca\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.079547 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.079367 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-trusted-ca-bundle\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.079547 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.079384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-oauth-serving-cert\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.080132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.080070 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-config\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.080274 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.080176 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-service-ca\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.080517 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.080492 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-oauth-serving-cert\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.080568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.080530 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-trusted-ca-bundle\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.082101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.082072 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-oauth-config\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.082227 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.082122 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-serving-cert\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.087177 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.087129 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8tq7\" (UniqueName: \"kubernetes.io/projected/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-kube-api-access-q8tq7\") pod \"console-9fcbf6949-x7q8x\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.162597 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.162562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:01.289007 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:01.288975 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9fcbf6949-x7q8x"] Jul 07 02:16:01.292046 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:16:01.292020 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc4427c_e383_4e5e_8dfb_42d28d24b6bb.slice/crio-7f3221cc6a33eeb07c6f2e8c7ed2a7d0f1d2d592985b2a90097ac33cd5bced87 WatchSource:0}: Error finding container 7f3221cc6a33eeb07c6f2e8c7ed2a7d0f1d2d592985b2a90097ac33cd5bced87: Status 404 returned error can't find the container with id 7f3221cc6a33eeb07c6f2e8c7ed2a7d0f1d2d592985b2a90097ac33cd5bced87 Jul 07 02:16:02.087467 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:02.087427 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fcbf6949-x7q8x" event={"ID":"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb","Type":"ContainerStarted","Data":"7cae48c63d920208b66c12479ceae8f68dd9d67e942b3a5a42d0f032e80c1df0"} Jul 07 02:16:02.087911 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:02.087478 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fcbf6949-x7q8x" event={"ID":"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb","Type":"ContainerStarted","Data":"7f3221cc6a33eeb07c6f2e8c7ed2a7d0f1d2d592985b2a90097ac33cd5bced87"} Jul 07 02:16:02.109703 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:02.109652 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9fcbf6949-x7q8x" podStartSLOduration=2.109637451 podStartE2EDuration="2.109637451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:16:02.10777973 +0000 UTC m=+378.858201389" watchObservedRunningTime="2026-07-07 02:16:02.109637451 +0000 UTC m=+378.860059104" Jul 07 02:16:11.162945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:11.162899 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:11.163431 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:11.162975 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:11.168192 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:11.168149 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:12.127987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:12.127956 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:16:47.461359 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.461320 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98"] Jul 07 02:16:47.464574 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.464556 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.467127 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.467107 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 02:16:47.468088 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.468075 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 02:16:47.468168 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.468124 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bjp67\"" Jul 07 02:16:47.473746 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.473723 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98"] Jul 07 02:16:47.566050 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.566013 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.566251 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.566077 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfhwd\" (UniqueName: \"kubernetes.io/projected/f8f2556c-9da0-4d08-9717-8303a6bec56c-kube-api-access-xfhwd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.566251 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.566218 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.666923 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.666885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.667094 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.666943 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.667094 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.666997 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfhwd\" (UniqueName: \"kubernetes.io/projected/f8f2556c-9da0-4d08-9717-8303a6bec56c-kube-api-access-xfhwd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.667294 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.667275 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.667409 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.667387 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.676211 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.676189 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfhwd\" (UniqueName: \"kubernetes.io/projected/f8f2556c-9da0-4d08-9717-8303a6bec56c-kube-api-access-xfhwd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.775347 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.775259 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:16:47.915234 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:47.915139 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98"] Jul 07 02:16:47.917754 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:16:47.917725 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f2556c_9da0_4d08_9717_8303a6bec56c.slice/crio-1de6c3821223804e12f06396e2121b69b935cbedfa6b6479053a1630de21d5ec WatchSource:0}: Error finding container 1de6c3821223804e12f06396e2121b69b935cbedfa6b6479053a1630de21d5ec: Status 404 returned error can't find the container with id 1de6c3821223804e12f06396e2121b69b935cbedfa6b6479053a1630de21d5ec Jul 07 02:16:48.256019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:48.255986 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" event={"ID":"f8f2556c-9da0-4d08-9717-8303a6bec56c","Type":"ContainerStarted","Data":"1de6c3821223804e12f06396e2121b69b935cbedfa6b6479053a1630de21d5ec"} Jul 07 02:16:56.290535 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:56.290495 2559 generic.go:356] "Generic (PLEG): container finished" podID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerID="9d08d0e01488b1c7664e019679b6cdea866d512aef95f8837a79daff580ab99b" exitCode=0 Jul 07 02:16:56.290939 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:56.290561 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" event={"ID":"f8f2556c-9da0-4d08-9717-8303a6bec56c","Type":"ContainerDied","Data":"9d08d0e01488b1c7664e019679b6cdea866d512aef95f8837a79daff580ab99b"} Jul 07 02:16:58.302244 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:58.302209 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" event={"ID":"f8f2556c-9da0-4d08-9717-8303a6bec56c","Type":"ContainerStarted","Data":"4204c89d6d6599f48c81a0724794a8cb69d430c56f7b5bb53377ac40dd03cd60"} Jul 07 02:16:59.308111 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:59.308080 2559 generic.go:356] "Generic (PLEG): container finished" podID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerID="4204c89d6d6599f48c81a0724794a8cb69d430c56f7b5bb53377ac40dd03cd60" exitCode=0 Jul 07 02:16:59.308553 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:16:59.308117 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" event={"ID":"f8f2556c-9da0-4d08-9717-8303a6bec56c","Type":"ContainerDied","Data":"4204c89d6d6599f48c81a0724794a8cb69d430c56f7b5bb53377ac40dd03cd60"} Jul 07 02:17:08.346900 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:08.346811 2559 generic.go:356] "Generic (PLEG): container finished" podID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerID="9716214d760865f649ba25be2aad679f3fb2035619409e2bd4543f790ba11cb7" exitCode=0 Jul 07 02:17:08.346900 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:08.346872 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" event={"ID":"f8f2556c-9da0-4d08-9717-8303a6bec56c","Type":"ContainerDied","Data":"9716214d760865f649ba25be2aad679f3fb2035619409e2bd4543f790ba11cb7"} Jul 07 02:17:09.495796 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.495769 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:17:09.664760 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.664734 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-util\") pod \"f8f2556c-9da0-4d08-9717-8303a6bec56c\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " Jul 07 02:17:09.664949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.664774 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-bundle\") pod \"f8f2556c-9da0-4d08-9717-8303a6bec56c\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " Jul 07 02:17:09.664949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.664804 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfhwd\" (UniqueName: \"kubernetes.io/projected/f8f2556c-9da0-4d08-9717-8303a6bec56c-kube-api-access-xfhwd\") pod \"f8f2556c-9da0-4d08-9717-8303a6bec56c\" (UID: \"f8f2556c-9da0-4d08-9717-8303a6bec56c\") " Jul 07 02:17:09.665517 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.665489 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-bundle" (OuterVolumeSpecName: "bundle") pod "f8f2556c-9da0-4d08-9717-8303a6bec56c" (UID: "f8f2556c-9da0-4d08-9717-8303a6bec56c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:17:09.666981 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.666958 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f2556c-9da0-4d08-9717-8303a6bec56c-kube-api-access-xfhwd" (OuterVolumeSpecName: "kube-api-access-xfhwd") pod "f8f2556c-9da0-4d08-9717-8303a6bec56c" (UID: "f8f2556c-9da0-4d08-9717-8303a6bec56c"). InnerVolumeSpecName "kube-api-access-xfhwd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:17:09.669510 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.669487 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-util" (OuterVolumeSpecName: "util") pod "f8f2556c-9da0-4d08-9717-8303a6bec56c" (UID: "f8f2556c-9da0-4d08-9717-8303a6bec56c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:17:09.766337 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.766295 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-util\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:09.766337 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.766328 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f2556c-9da0-4d08-9717-8303a6bec56c-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:09.766337 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:09.766339 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfhwd\" (UniqueName: \"kubernetes.io/projected/f8f2556c-9da0-4d08-9717-8303a6bec56c-kube-api-access-xfhwd\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:10.361511 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:10.361478 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" event={"ID":"f8f2556c-9da0-4d08-9717-8303a6bec56c","Type":"ContainerDied","Data":"1de6c3821223804e12f06396e2121b69b935cbedfa6b6479053a1630de21d5ec"} Jul 07 02:17:10.361673 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:10.361521 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409st98" Jul 07 02:17:10.361673 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:10.361546 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1de6c3821223804e12f06396e2121b69b935cbedfa6b6479053a1630de21d5ec" Jul 07 02:17:19.180505 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.180464 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp"] Jul 07 02:17:19.181100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.180885 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerName="util" Jul 07 02:17:19.181100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.180903 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerName="util" Jul 07 02:17:19.181100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.180935 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerName="extract" Jul 07 02:17:19.181100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.180944 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerName="extract" Jul 07 02:17:19.181100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.180953 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerName="pull" Jul 07 02:17:19.181100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.180964 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerName="pull" Jul 07 02:17:19.181100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.181050 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f2556c-9da0-4d08-9717-8303a6bec56c" containerName="extract" Jul 07 02:17:19.185386 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.185366 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.188036 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.188019 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7jq2m\"" Jul 07 02:17:19.188220 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.188201 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 02:17:19.188463 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.188448 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 02:17:19.201996 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.201967 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp"] Jul 07 02:17:19.242945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.242915 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdnpq\" (UniqueName: \"kubernetes.io/projected/03a52466-8a87-4db2-92a8-573584dbe3b2-kube-api-access-wdnpq\") pod \"cert-manager-operator-controller-manager-55c444bf86-4bbpp\" (UID: \"03a52466-8a87-4db2-92a8-573584dbe3b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.243116 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.243041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03a52466-8a87-4db2-92a8-573584dbe3b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4bbpp\" (UID: \"03a52466-8a87-4db2-92a8-573584dbe3b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.343661 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.343625 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdnpq\" (UniqueName: \"kubernetes.io/projected/03a52466-8a87-4db2-92a8-573584dbe3b2-kube-api-access-wdnpq\") pod \"cert-manager-operator-controller-manager-55c444bf86-4bbpp\" (UID: \"03a52466-8a87-4db2-92a8-573584dbe3b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.343825 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.343701 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03a52466-8a87-4db2-92a8-573584dbe3b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4bbpp\" (UID: \"03a52466-8a87-4db2-92a8-573584dbe3b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.343999 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.343985 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03a52466-8a87-4db2-92a8-573584dbe3b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4bbpp\" (UID: \"03a52466-8a87-4db2-92a8-573584dbe3b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.354399 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.354367 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdnpq\" (UniqueName: \"kubernetes.io/projected/03a52466-8a87-4db2-92a8-573584dbe3b2-kube-api-access-wdnpq\") pod \"cert-manager-operator-controller-manager-55c444bf86-4bbpp\" (UID: \"03a52466-8a87-4db2-92a8-573584dbe3b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.493898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.493817 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" Jul 07 02:17:19.629831 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:19.629797 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp"] Jul 07 02:17:19.633460 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:17:19.633423 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a52466_8a87_4db2_92a8_573584dbe3b2.slice/crio-86df8325309b7515f678155416c2105988580101472e5ee7af7286d0838e8fb2 WatchSource:0}: Error finding container 86df8325309b7515f678155416c2105988580101472e5ee7af7286d0838e8fb2: Status 404 returned error can't find the container with id 86df8325309b7515f678155416c2105988580101472e5ee7af7286d0838e8fb2 Jul 07 02:17:20.402778 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:20.402713 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" event={"ID":"03a52466-8a87-4db2-92a8-573584dbe3b2","Type":"ContainerStarted","Data":"86df8325309b7515f678155416c2105988580101472e5ee7af7286d0838e8fb2"} Jul 07 02:17:21.409402 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:21.409363 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" event={"ID":"03a52466-8a87-4db2-92a8-573584dbe3b2","Type":"ContainerStarted","Data":"d4678b3231c07409e5096f8b26e637375114aef8430bd67cf5992fabaf3aa891"} Jul 07 02:17:21.432954 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:21.432832 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4bbpp" podStartSLOduration=0.896236874 podStartE2EDuration="2.432812393s" totalImagesPullingTime="1.536575519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:17:21.431295726 +0000 UTC m=+458.181717371" watchObservedRunningTime="2026-07-07 02:17:21.432812393 +0000 UTC m=+458.183234046" Jul 07 02:17:23.848608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.848572 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h"] Jul 07 02:17:23.851366 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.851350 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.853701 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.853678 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 02:17:23.853822 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.853714 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 02:17:23.854621 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.854606 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bjp67\"" Jul 07 02:17:23.859448 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.859425 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h"] Jul 07 02:17:23.883343 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.883310 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.883343 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.883341 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmj4s\" (UniqueName: \"kubernetes.io/projected/b7c30b98-7ef6-4c43-8713-86c2241896dd-kube-api-access-lmj4s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.883575 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.883549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.984441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.984408 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.984441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.984449 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.984704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.984472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmj4s\" (UniqueName: \"kubernetes.io/projected/b7c30b98-7ef6-4c43-8713-86c2241896dd-kube-api-access-lmj4s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.984871 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.984846 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.984937 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.984903 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:23.992968 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:23.992942 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmj4s\" (UniqueName: \"kubernetes.io/projected/b7c30b98-7ef6-4c43-8713-86c2241896dd-kube-api-access-lmj4s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:24.161382 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:24.161349 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:24.290669 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:24.290647 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h"] Jul 07 02:17:24.293104 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:17:24.293075 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c30b98_7ef6_4c43_8713_86c2241896dd.slice/crio-05f85a49e4c3e6587042eb72c8498662d1879b6d08e2f5078cd464138d5474d8 WatchSource:0}: Error finding container 05f85a49e4c3e6587042eb72c8498662d1879b6d08e2f5078cd464138d5474d8: Status 404 returned error can't find the container with id 05f85a49e4c3e6587042eb72c8498662d1879b6d08e2f5078cd464138d5474d8 Jul 07 02:17:24.428600 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:24.428526 2559 generic.go:356] "Generic (PLEG): container finished" podID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerID="8c0777505f00d8af5d5890e80aa4fad501932cc7a0c9f84d2ef50ecbba69ac35" exitCode=0 Jul 07 02:17:24.428600 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:24.428567 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" event={"ID":"b7c30b98-7ef6-4c43-8713-86c2241896dd","Type":"ContainerDied","Data":"8c0777505f00d8af5d5890e80aa4fad501932cc7a0c9f84d2ef50ecbba69ac35"} Jul 07 02:17:24.428777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:24.428627 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" event={"ID":"b7c30b98-7ef6-4c43-8713-86c2241896dd","Type":"ContainerStarted","Data":"05f85a49e4c3e6587042eb72c8498662d1879b6d08e2f5078cd464138d5474d8"} Jul 07 02:17:25.012745 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.012709 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7jz6j"] Jul 07 02:17:25.015008 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.014986 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.017324 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.017301 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 02:17:25.017471 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.017429 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bdrqd\"" Jul 07 02:17:25.018344 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.018326 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 02:17:25.023404 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.023379 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7jz6j"] Jul 07 02:17:25.095437 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.095407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2be9f40-8cd7-410e-a32a-d133a526f8f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7jz6j\" (UID: \"e2be9f40-8cd7-410e-a32a-d133a526f8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.095437 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.095442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwll2\" (UniqueName: \"kubernetes.io/projected/e2be9f40-8cd7-410e-a32a-d133a526f8f5-kube-api-access-zwll2\") pod \"cert-manager-webhook-6486ffd997-7jz6j\" (UID: \"e2be9f40-8cd7-410e-a32a-d133a526f8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.196629 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.196590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2be9f40-8cd7-410e-a32a-d133a526f8f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7jz6j\" (UID: \"e2be9f40-8cd7-410e-a32a-d133a526f8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.196629 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.196632 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwll2\" (UniqueName: \"kubernetes.io/projected/e2be9f40-8cd7-410e-a32a-d133a526f8f5-kube-api-access-zwll2\") pod \"cert-manager-webhook-6486ffd997-7jz6j\" (UID: \"e2be9f40-8cd7-410e-a32a-d133a526f8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.205723 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.205681 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2be9f40-8cd7-410e-a32a-d133a526f8f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7jz6j\" (UID: \"e2be9f40-8cd7-410e-a32a-d133a526f8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.205912 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.205747 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwll2\" (UniqueName: \"kubernetes.io/projected/e2be9f40-8cd7-410e-a32a-d133a526f8f5-kube-api-access-zwll2\") pod \"cert-manager-webhook-6486ffd997-7jz6j\" (UID: \"e2be9f40-8cd7-410e-a32a-d133a526f8f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.334897 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.334812 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:25.491767 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:25.491741 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7jz6j"] Jul 07 02:17:25.494408 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:17:25.494376 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2be9f40_8cd7_410e_a32a_d133a526f8f5.slice/crio-a23c256f41198c3dde1277e6f0f42400af9901c013d3d65fb01183f19ed4ab4f WatchSource:0}: Error finding container a23c256f41198c3dde1277e6f0f42400af9901c013d3d65fb01183f19ed4ab4f: Status 404 returned error can't find the container with id a23c256f41198c3dde1277e6f0f42400af9901c013d3d65fb01183f19ed4ab4f Jul 07 02:17:26.440820 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:26.440781 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" event={"ID":"b7c30b98-7ef6-4c43-8713-86c2241896dd","Type":"ContainerStarted","Data":"ed8867c0340151fb0f7e0a3ac8befa943da57e6836a1a66566a8f6db0d4beaac"} Jul 07 02:17:26.442054 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:26.442030 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" event={"ID":"e2be9f40-8cd7-410e-a32a-d133a526f8f5","Type":"ContainerStarted","Data":"a23c256f41198c3dde1277e6f0f42400af9901c013d3d65fb01183f19ed4ab4f"} Jul 07 02:17:27.448443 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:27.448406 2559 generic.go:356] "Generic (PLEG): container finished" podID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerID="ed8867c0340151fb0f7e0a3ac8befa943da57e6836a1a66566a8f6db0d4beaac" exitCode=0 Jul 07 02:17:27.448806 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:27.448475 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" event={"ID":"b7c30b98-7ef6-4c43-8713-86c2241896dd","Type":"ContainerDied","Data":"ed8867c0340151fb0f7e0a3ac8befa943da57e6836a1a66566a8f6db0d4beaac"} Jul 07 02:17:28.454937 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:28.454892 2559 generic.go:356] "Generic (PLEG): container finished" podID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerID="ffab20fdbc06171548dc641466fd0e7828edbb26509f0c0ec2491cb95b57f9ea" exitCode=0 Jul 07 02:17:28.454937 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:28.454936 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" event={"ID":"b7c30b98-7ef6-4c43-8713-86c2241896dd","Type":"ContainerDied","Data":"ffab20fdbc06171548dc641466fd0e7828edbb26509f0c0ec2491cb95b57f9ea"} Jul 07 02:17:29.589101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.589072 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:29.632570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.632542 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-util\") pod \"b7c30b98-7ef6-4c43-8713-86c2241896dd\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " Jul 07 02:17:29.632726 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.632584 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmj4s\" (UniqueName: \"kubernetes.io/projected/b7c30b98-7ef6-4c43-8713-86c2241896dd-kube-api-access-lmj4s\") pod \"b7c30b98-7ef6-4c43-8713-86c2241896dd\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " Jul 07 02:17:29.632726 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.632644 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-bundle\") pod \"b7c30b98-7ef6-4c43-8713-86c2241896dd\" (UID: \"b7c30b98-7ef6-4c43-8713-86c2241896dd\") " Jul 07 02:17:29.633061 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.633039 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-bundle" (OuterVolumeSpecName: "bundle") pod "b7c30b98-7ef6-4c43-8713-86c2241896dd" (UID: "b7c30b98-7ef6-4c43-8713-86c2241896dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:17:29.634740 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.634709 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c30b98-7ef6-4c43-8713-86c2241896dd-kube-api-access-lmj4s" (OuterVolumeSpecName: "kube-api-access-lmj4s") pod "b7c30b98-7ef6-4c43-8713-86c2241896dd" (UID: "b7c30b98-7ef6-4c43-8713-86c2241896dd"). InnerVolumeSpecName "kube-api-access-lmj4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:17:29.637037 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.636998 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-util" (OuterVolumeSpecName: "util") pod "b7c30b98-7ef6-4c43-8713-86c2241896dd" (UID: "b7c30b98-7ef6-4c43-8713-86c2241896dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:17:29.733277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.733179 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:29.733277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.733214 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7c30b98-7ef6-4c43-8713-86c2241896dd-util\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:29.733277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:29.733223 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmj4s\" (UniqueName: \"kubernetes.io/projected/b7c30b98-7ef6-4c43-8713-86c2241896dd-kube-api-access-lmj4s\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:30.465123 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:30.465094 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" Jul 07 02:17:30.465123 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:30.465105 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdr65h" event={"ID":"b7c30b98-7ef6-4c43-8713-86c2241896dd","Type":"ContainerDied","Data":"05f85a49e4c3e6587042eb72c8498662d1879b6d08e2f5078cd464138d5474d8"} Jul 07 02:17:30.465354 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:30.465145 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05f85a49e4c3e6587042eb72c8498662d1879b6d08e2f5078cd464138d5474d8" Jul 07 02:17:33.479387 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:33.479352 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" event={"ID":"e2be9f40-8cd7-410e-a32a-d133a526f8f5","Type":"ContainerStarted","Data":"644163a7411aa6cddbfe71961ca1c128feccad452ea4d327e73a37f83f7ea29f"} Jul 07 02:17:33.479744 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:33.479397 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:33.495748 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:33.495696 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" podStartSLOduration=1.6575114069999999 podStartE2EDuration="9.495679532s" totalImagesPullingTime="7.838168125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:17:33.494666831 +0000 UTC m=+470.245088485" watchObservedRunningTime="2026-07-07 02:17:33.495679532 +0000 UTC m=+470.246101185" Jul 07 02:17:39.485718 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:39.485685 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7jz6j" Jul 07 02:17:47.718037 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718001 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g"] Jul 07 02:17:47.718527 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718326 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerName="util" Jul 07 02:17:47.718527 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718338 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerName="util" Jul 07 02:17:47.718527 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718350 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerName="extract" Jul 07 02:17:47.718527 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718355 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerName="extract" Jul 07 02:17:47.718527 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718365 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerName="pull" Jul 07 02:17:47.718527 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718370 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerName="pull" Jul 07 02:17:47.718527 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.718425 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7c30b98-7ef6-4c43-8713-86c2241896dd" containerName="extract" Jul 07 02:17:47.722927 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.722910 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.725210 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.725185 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bjp67\"" Jul 07 02:17:47.725336 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.725212 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 02:17:47.726011 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.725994 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 02:17:47.729650 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.729627 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g"] Jul 07 02:17:47.893205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.893145 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.893205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.893213 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.893417 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.893275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlkvl\" (UniqueName: \"kubernetes.io/projected/6377affd-908f-4f1d-a6ad-cce27e1ba155-kube-api-access-wlkvl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.994622 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.994532 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.994622 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.994576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.994817 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.994713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlkvl\" (UniqueName: \"kubernetes.io/projected/6377affd-908f-4f1d-a6ad-cce27e1ba155-kube-api-access-wlkvl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.994986 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.994965 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:47.995021 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:47.994981 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:48.002657 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:48.002620 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlkvl\" (UniqueName: \"kubernetes.io/projected/6377affd-908f-4f1d-a6ad-cce27e1ba155-kube-api-access-wlkvl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:48.032701 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:48.032664 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:48.163840 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:48.163815 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g"] Jul 07 02:17:48.166787 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:17:48.166759 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6377affd_908f_4f1d_a6ad_cce27e1ba155.slice/crio-783d71b2faa5515cf34df786276ea35c53ea3c7114d3583b4e17f68bbcdf435b WatchSource:0}: Error finding container 783d71b2faa5515cf34df786276ea35c53ea3c7114d3583b4e17f68bbcdf435b: Status 404 returned error can't find the container with id 783d71b2faa5515cf34df786276ea35c53ea3c7114d3583b4e17f68bbcdf435b Jul 07 02:17:48.543507 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:48.543413 2559 generic.go:356] "Generic (PLEG): container finished" podID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerID="c1d29d01c4529f519f1b15793e87b85f44ef19c0ffbc3eae9e8edd2928149bd8" exitCode=0 Jul 07 02:17:48.543507 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:48.543461 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" event={"ID":"6377affd-908f-4f1d-a6ad-cce27e1ba155","Type":"ContainerDied","Data":"c1d29d01c4529f519f1b15793e87b85f44ef19c0ffbc3eae9e8edd2928149bd8"} Jul 07 02:17:48.543685 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:48.543511 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" event={"ID":"6377affd-908f-4f1d-a6ad-cce27e1ba155","Type":"ContainerStarted","Data":"783d71b2faa5515cf34df786276ea35c53ea3c7114d3583b4e17f68bbcdf435b"} Jul 07 02:17:49.549433 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:49.549393 2559 generic.go:356] "Generic (PLEG): container finished" podID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerID="7bb0dcffc4a35b497f527d3593be1cb100566bc6484fff99fa1c60f65279dc5c" exitCode=0 Jul 07 02:17:49.549811 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:49.549479 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" event={"ID":"6377affd-908f-4f1d-a6ad-cce27e1ba155","Type":"ContainerDied","Data":"7bb0dcffc4a35b497f527d3593be1cb100566bc6484fff99fa1c60f65279dc5c"} Jul 07 02:17:50.556668 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:50.556636 2559 generic.go:356] "Generic (PLEG): container finished" podID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerID="55546e0ae11a841a794f58030fe2869a42f37f8ce4afd90a36a2c808737d4a51" exitCode=0 Jul 07 02:17:50.557035 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:50.556707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" event={"ID":"6377affd-908f-4f1d-a6ad-cce27e1ba155","Type":"ContainerDied","Data":"55546e0ae11a841a794f58030fe2869a42f37f8ce4afd90a36a2c808737d4a51"} Jul 07 02:17:51.695711 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.695689 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:51.829987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.829916 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-util\") pod \"6377affd-908f-4f1d-a6ad-cce27e1ba155\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " Jul 07 02:17:51.829987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.829973 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-bundle\") pod \"6377affd-908f-4f1d-a6ad-cce27e1ba155\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " Jul 07 02:17:51.830231 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.830002 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlkvl\" (UniqueName: \"kubernetes.io/projected/6377affd-908f-4f1d-a6ad-cce27e1ba155-kube-api-access-wlkvl\") pod \"6377affd-908f-4f1d-a6ad-cce27e1ba155\" (UID: \"6377affd-908f-4f1d-a6ad-cce27e1ba155\") " Jul 07 02:17:51.830734 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.830710 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-bundle" (OuterVolumeSpecName: "bundle") pod "6377affd-908f-4f1d-a6ad-cce27e1ba155" (UID: "6377affd-908f-4f1d-a6ad-cce27e1ba155"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:17:51.832034 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.832012 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6377affd-908f-4f1d-a6ad-cce27e1ba155-kube-api-access-wlkvl" (OuterVolumeSpecName: "kube-api-access-wlkvl") pod "6377affd-908f-4f1d-a6ad-cce27e1ba155" (UID: "6377affd-908f-4f1d-a6ad-cce27e1ba155"). InnerVolumeSpecName "kube-api-access-wlkvl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:17:51.835249 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.835214 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-util" (OuterVolumeSpecName: "util") pod "6377affd-908f-4f1d-a6ad-cce27e1ba155" (UID: "6377affd-908f-4f1d-a6ad-cce27e1ba155"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:17:51.930521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.930487 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-util\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:51.930521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.930515 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6377affd-908f-4f1d-a6ad-cce27e1ba155-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:51.930521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:51.930524 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlkvl\" (UniqueName: \"kubernetes.io/projected/6377affd-908f-4f1d-a6ad-cce27e1ba155-kube-api-access-wlkvl\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:17:52.566756 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:52.566723 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" event={"ID":"6377affd-908f-4f1d-a6ad-cce27e1ba155","Type":"ContainerDied","Data":"783d71b2faa5515cf34df786276ea35c53ea3c7114d3583b4e17f68bbcdf435b"} Jul 07 02:17:52.566898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:52.566771 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="783d71b2faa5515cf34df786276ea35c53ea3c7114d3583b4e17f68bbcdf435b" Jul 07 02:17:52.566898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:52.566774 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr9t5g" Jul 07 02:17:59.756088 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756054 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh"] Jul 07 02:17:59.756481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756359 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerName="pull" Jul 07 02:17:59.756481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756369 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerName="pull" Jul 07 02:17:59.756481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756388 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerName="extract" Jul 07 02:17:59.756481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756394 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerName="extract" Jul 07 02:17:59.756481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756405 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerName="util" Jul 07 02:17:59.756481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756410 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerName="util" Jul 07 02:17:59.756481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.756458 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6377affd-908f-4f1d-a6ad-cce27e1ba155" containerName="extract" Jul 07 02:17:59.759623 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.759606 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.762464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.762441 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 02:17:59.762464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.762456 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 02:17:59.762464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.762479 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 02:17:59.762666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.762444 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 02:17:59.762666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.762456 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6ghdc\"" Jul 07 02:17:59.774678 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.774659 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh"] Jul 07 02:17:59.797164 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.797127 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58bcx\" (UniqueName: \"kubernetes.io/projected/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-kube-api-access-58bcx\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.797289 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.797182 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-webhook-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.797289 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.797204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-manifests\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.797289 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.797225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-tmp\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.797289 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.797273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.898559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.898519 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58bcx\" (UniqueName: \"kubernetes.io/projected/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-kube-api-access-58bcx\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.898559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.898559 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-webhook-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.898800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.898576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-manifests\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.898800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.898683 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-tmp\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.898800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.898745 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.898959 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.898933 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-manifests\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.901212 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.901186 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.901389 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.901366 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-tmp\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.901582 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.901562 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-webhook-cert\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:17:59.910684 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:17:59.910657 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58bcx\" (UniqueName: \"kubernetes.io/projected/6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52-kube-api-access-58bcx\") pod \"opendatahub-operator-controller-manager-76797ccd4c-k9tkh\" (UID: \"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52\") " pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:18:00.071371 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:00.071272 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:18:00.203337 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:00.203270 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh"] Jul 07 02:18:00.205602 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:18:00.205580 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf320b7_9a36_4cfa_b1c6_d04e0c7e8c52.slice/crio-ae068f76e182c1ff56ac76260aa4dc4cf35f8ef0033cd211e09febe696527cff WatchSource:0}: Error finding container ae068f76e182c1ff56ac76260aa4dc4cf35f8ef0033cd211e09febe696527cff: Status 404 returned error can't find the container with id ae068f76e182c1ff56ac76260aa4dc4cf35f8ef0033cd211e09febe696527cff Jul 07 02:18:00.602175 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:00.602121 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" event={"ID":"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52","Type":"ContainerStarted","Data":"ae068f76e182c1ff56ac76260aa4dc4cf35f8ef0033cd211e09febe696527cff"} Jul 07 02:18:03.623940 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:03.623859 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" event={"ID":"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52","Type":"ContainerStarted","Data":"3bdfa496546913457913d321626df614675f4aac7311ac7f5f573dc5a6ce8c39"} Jul 07 02:18:04.629833 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:04.629743 2559 generic.go:356] "Generic (PLEG): container finished" podID="6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52" containerID="3bdfa496546913457913d321626df614675f4aac7311ac7f5f573dc5a6ce8c39" exitCode=0 Jul 07 02:18:04.630350 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:04.629827 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" event={"ID":"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52","Type":"ContainerDied","Data":"3bdfa496546913457913d321626df614675f4aac7311ac7f5f573dc5a6ce8c39"} Jul 07 02:18:05.637366 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:05.637329 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" event={"ID":"6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52","Type":"ContainerStarted","Data":"0a8d5e4670f133cadf6b2792f13b86c10046d45692e720aa80ac4ce8fdaba274"} Jul 07 02:18:05.637838 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:05.637450 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:18:05.660467 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:05.660414 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" podStartSLOduration=2.13357303 podStartE2EDuration="6.660399932s" totalImagesPullingTime="3.457849898s" totalInitContainerRuntime="1.068977004s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:17:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:05.658270535 +0000 UTC m=+502.408692189" watchObservedRunningTime="2026-07-07 02:18:05.660399932 +0000 UTC m=+502.410821601" Jul 07 02:18:16.644197 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:16.644143 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76797ccd4c-k9tkh" Jul 07 02:18:18.386684 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.386648 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7"] Jul 07 02:18:18.397708 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.397684 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.399652 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.399613 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7"] Jul 07 02:18:18.401229 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.401205 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f855d\"" Jul 07 02:18:18.401446 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.401215 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 02:18:18.401535 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.401234 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 02:18:18.401535 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.401234 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 02:18:18.401535 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.401242 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 02:18:18.401699 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.401249 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 02:18:18.466509 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.466479 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b58fbd5-e83e-4278-9d7d-8e054d66066c-manager-config\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.466509 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.466511 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b58fbd5-e83e-4278-9d7d-8e054d66066c-cert\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.466721 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.466536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b58fbd5-e83e-4278-9d7d-8e054d66066c-metrics-cert\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.466721 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.466624 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8992\" (UniqueName: \"kubernetes.io/projected/8b58fbd5-e83e-4278-9d7d-8e054d66066c-kube-api-access-f8992\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.568045 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.568009 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8992\" (UniqueName: \"kubernetes.io/projected/8b58fbd5-e83e-4278-9d7d-8e054d66066c-kube-api-access-f8992\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.568274 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.568086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b58fbd5-e83e-4278-9d7d-8e054d66066c-manager-config\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.568274 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.568112 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b58fbd5-e83e-4278-9d7d-8e054d66066c-cert\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.568274 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.568137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b58fbd5-e83e-4278-9d7d-8e054d66066c-metrics-cert\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.568884 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.568854 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b58fbd5-e83e-4278-9d7d-8e054d66066c-manager-config\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.570832 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.570780 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b58fbd5-e83e-4278-9d7d-8e054d66066c-metrics-cert\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.571495 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.571471 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b58fbd5-e83e-4278-9d7d-8e054d66066c-cert\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.583983 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.583955 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8992\" (UniqueName: \"kubernetes.io/projected/8b58fbd5-e83e-4278-9d7d-8e054d66066c-kube-api-access-f8992\") pod \"lws-controller-manager-5bd684c58c-7l2j7\" (UID: \"8b58fbd5-e83e-4278-9d7d-8e054d66066c\") " pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.707847 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.707821 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:18.845608 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:18.845577 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7"] Jul 07 02:18:18.847856 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:18:18.847827 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b58fbd5_e83e_4278_9d7d_8e054d66066c.slice/crio-a960414a818ea30fde403c346b71b7115f0fc47a9aadbfea87b38f75ddb85c90 WatchSource:0}: Error finding container a960414a818ea30fde403c346b71b7115f0fc47a9aadbfea87b38f75ddb85c90: Status 404 returned error can't find the container with id a960414a818ea30fde403c346b71b7115f0fc47a9aadbfea87b38f75ddb85c90 Jul 07 02:18:19.703224 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:19.703192 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" event={"ID":"8b58fbd5-e83e-4278-9d7d-8e054d66066c","Type":"ContainerStarted","Data":"a960414a818ea30fde403c346b71b7115f0fc47a9aadbfea87b38f75ddb85c90"} Jul 07 02:18:20.709212 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:20.709145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" event={"ID":"8b58fbd5-e83e-4278-9d7d-8e054d66066c","Type":"ContainerStarted","Data":"555e97714370280da0ca333933912247e3041ccee72f68ea6fb9c0c39edf832c"} Jul 07 02:18:20.709590 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:20.709229 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:20.726574 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:20.726525 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" podStartSLOduration=1.187725515 podStartE2EDuration="2.726509802s" totalImagesPullingTime="1.538784287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:20.724357896 +0000 UTC m=+517.474779549" watchObservedRunningTime="2026-07-07 02:18:20.726509802 +0000 UTC m=+517.476931504" Jul 07 02:18:31.715785 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:31.715754 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bd684c58c-7l2j7" Jul 07 02:18:36.970563 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:36.970483 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl"] Jul 07 02:18:36.973424 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:36.973388 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:36.977899 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:36.977837 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 02:18:36.977899 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:36.977848 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gnq7r\"" Jul 07 02:18:36.985138 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:36.985116 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl"] Jul 07 02:18:37.133679 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133643 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133697 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133719 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h7kq\" (UniqueName: \"kubernetes.io/projected/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-kube-api-access-9h7kq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133739 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133801 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133848 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133886 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.133934 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.133907 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.234800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.234709 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.234800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.234754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.234800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.234774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.234800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.234799 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.234977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235031 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235104 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235127 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h7kq\" (UniqueName: \"kubernetes.io/projected/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-kube-api-access-9h7kq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235192 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235361 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235477 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235402 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235477 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235582 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.235747 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.235726 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.237323 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.237303 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.237435 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.237412 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.242600 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.242580 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.242930 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.242898 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h7kq\" (UniqueName: \"kubernetes.io/projected/f846a0a8-35c2-461f-9fe1-dd85bed1c26a-kube-api-access-9h7kq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45wlpl\" (UID: \"f846a0a8-35c2-461f-9fe1-dd85bed1c26a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.286622 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.286586 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:37.419129 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.419098 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl"] Jul 07 02:18:37.420476 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:18:37.420449 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf846a0a8_35c2_461f_9fe1_dd85bed1c26a.slice/crio-9270a71111d48fce8de3d8d231952b49ac7e0b1ab3927ce0852b92764fde2972 WatchSource:0}: Error finding container 9270a71111d48fce8de3d8d231952b49ac7e0b1ab3927ce0852b92764fde2972: Status 404 returned error can't find the container with id 9270a71111d48fce8de3d8d231952b49ac7e0b1ab3927ce0852b92764fde2972 Jul 07 02:18:37.785590 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:37.785558 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" event={"ID":"f846a0a8-35c2-461f-9fe1-dd85bed1c26a","Type":"ContainerStarted","Data":"9270a71111d48fce8de3d8d231952b49ac7e0b1ab3927ce0852b92764fde2972"} Jul 07 02:18:40.153167 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:40.153117 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:18:40.153430 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:40.153213 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:18:40.153430 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:40.153249 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:18:40.802417 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:40.802385 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" event={"ID":"f846a0a8-35c2-461f-9fe1-dd85bed1c26a","Type":"ContainerStarted","Data":"6653b95d501c0668b01fce2411f4d2467eb73ff6f5892095f6e5926640a3db64"} Jul 07 02:18:40.824101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:40.824051 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" podStartSLOduration=2.093799876 podStartE2EDuration="4.824037153s" totalImagesPullingTime="2.730237277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:18:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:18:40.822644812 +0000 UTC m=+537.573066466" watchObservedRunningTime="2026-07-07 02:18:40.824037153 +0000 UTC m=+537.574458806" Jul 07 02:18:41.287802 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:41.287768 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:41.292584 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:41.292557 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:41.807738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:41.807705 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:18:41.808765 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:18:41.808744 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45wlpl" Jul 07 02:19:20.733590 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.733551 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lc22c"] Jul 07 02:19:20.742280 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.742253 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" Jul 07 02:19:20.744918 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.744889 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 02:19:20.746048 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.745923 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 02:19:20.746453 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.746424 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-q7wt9\"" Jul 07 02:19:20.759802 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.759775 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lc22c"] Jul 07 02:19:20.806292 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.806257 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvxpc\" (UniqueName: \"kubernetes.io/projected/fc444786-9036-4e36-9f80-5e6e7f085650-kube-api-access-pvxpc\") pod \"kuadrant-operator-catalog-lc22c\" (UID: \"fc444786-9036-4e36-9f80-5e6e7f085650\") " pod="kuadrant-system/kuadrant-operator-catalog-lc22c" Jul 07 02:19:20.907719 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.907686 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvxpc\" (UniqueName: \"kubernetes.io/projected/fc444786-9036-4e36-9f80-5e6e7f085650-kube-api-access-pvxpc\") pod \"kuadrant-operator-catalog-lc22c\" (UID: \"fc444786-9036-4e36-9f80-5e6e7f085650\") " pod="kuadrant-system/kuadrant-operator-catalog-lc22c" Jul 07 02:19:20.916030 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:20.915991 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvxpc\" (UniqueName: \"kubernetes.io/projected/fc444786-9036-4e36-9f80-5e6e7f085650-kube-api-access-pvxpc\") pod \"kuadrant-operator-catalog-lc22c\" (UID: \"fc444786-9036-4e36-9f80-5e6e7f085650\") " pod="kuadrant-system/kuadrant-operator-catalog-lc22c" Jul 07 02:19:21.060407 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.060327 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" Jul 07 02:19:21.098451 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.098410 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lc22c"] Jul 07 02:19:21.191104 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.191079 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lc22c"] Jul 07 02:19:21.193262 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:21.193225 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc444786_9036_4e36_9f80_5e6e7f085650.slice/crio-bec598b947dfd6c6c990762ec2ccfcee8cc33ee4586db27a1e204781f66d53ac WatchSource:0}: Error finding container bec598b947dfd6c6c990762ec2ccfcee8cc33ee4586db27a1e204781f66d53ac: Status 404 returned error can't find the container with id bec598b947dfd6c6c990762ec2ccfcee8cc33ee4586db27a1e204781f66d53ac Jul 07 02:19:21.306768 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.306734 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ptn4f"] Jul 07 02:19:21.311277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.311234 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:21.316586 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.316562 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ptn4f"] Jul 07 02:19:21.412132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.412093 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcmc6\" (UniqueName: \"kubernetes.io/projected/3a5507c1-54ad-41cf-9a10-4667987fdf4d-kube-api-access-jcmc6\") pod \"kuadrant-operator-catalog-ptn4f\" (UID: \"3a5507c1-54ad-41cf-9a10-4667987fdf4d\") " pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:21.513461 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.513424 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcmc6\" (UniqueName: \"kubernetes.io/projected/3a5507c1-54ad-41cf-9a10-4667987fdf4d-kube-api-access-jcmc6\") pod \"kuadrant-operator-catalog-ptn4f\" (UID: \"3a5507c1-54ad-41cf-9a10-4667987fdf4d\") " pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:21.521144 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.521114 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcmc6\" (UniqueName: \"kubernetes.io/projected/3a5507c1-54ad-41cf-9a10-4667987fdf4d-kube-api-access-jcmc6\") pod \"kuadrant-operator-catalog-ptn4f\" (UID: \"3a5507c1-54ad-41cf-9a10-4667987fdf4d\") " pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:21.621520 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.621427 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:21.769372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.769338 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ptn4f"] Jul 07 02:19:21.772542 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:21.772508 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5507c1_54ad_41cf_9a10_4667987fdf4d.slice/crio-953b5dc6c0da6cfae72dcab27036cd1ac3572797c8050b4ed10822e5faeb165d WatchSource:0}: Error finding container 953b5dc6c0da6cfae72dcab27036cd1ac3572797c8050b4ed10822e5faeb165d: Status 404 returned error can't find the container with id 953b5dc6c0da6cfae72dcab27036cd1ac3572797c8050b4ed10822e5faeb165d Jul 07 02:19:21.987788 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.987734 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" event={"ID":"fc444786-9036-4e36-9f80-5e6e7f085650","Type":"ContainerStarted","Data":"bec598b947dfd6c6c990762ec2ccfcee8cc33ee4586db27a1e204781f66d53ac"} Jul 07 02:19:21.989171 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:21.989129 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" event={"ID":"3a5507c1-54ad-41cf-9a10-4667987fdf4d","Type":"ContainerStarted","Data":"953b5dc6c0da6cfae72dcab27036cd1ac3572797c8050b4ed10822e5faeb165d"} Jul 07 02:19:24.001779 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.001736 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" event={"ID":"3a5507c1-54ad-41cf-9a10-4667987fdf4d","Type":"ContainerStarted","Data":"655e6c0f39eb66526b4283253e5d61ec0c9714904793009636e6502f1616335b"} Jul 07 02:19:24.003274 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.003250 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" event={"ID":"fc444786-9036-4e36-9f80-5e6e7f085650","Type":"ContainerStarted","Data":"a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667"} Jul 07 02:19:24.003411 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.003292 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" podUID="fc444786-9036-4e36-9f80-5e6e7f085650" containerName="registry-server" containerID="cri-o://a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667" gracePeriod=2 Jul 07 02:19:24.018040 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.017993 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" podStartSLOduration=1.240996906 podStartE2EDuration="3.017971061s" totalImagesPullingTime="1.776974155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:24.016886489 +0000 UTC m=+580.767308143" watchObservedRunningTime="2026-07-07 02:19:24.017971061 +0000 UTC m=+580.768392716" Jul 07 02:19:24.030461 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.030331 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" podStartSLOduration=1.676889118 podStartE2EDuration="4.03031203s" totalImagesPullingTime="2.353422912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:24.029956886 +0000 UTC m=+580.780378541" watchObservedRunningTime="2026-07-07 02:19:24.03031203 +0000 UTC m=+580.780733685" Jul 07 02:19:24.243074 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.243048 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" Jul 07 02:19:24.346465 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.346376 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvxpc\" (UniqueName: \"kubernetes.io/projected/fc444786-9036-4e36-9f80-5e6e7f085650-kube-api-access-pvxpc\") pod \"fc444786-9036-4e36-9f80-5e6e7f085650\" (UID: \"fc444786-9036-4e36-9f80-5e6e7f085650\") " Jul 07 02:19:24.348550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.348527 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc444786-9036-4e36-9f80-5e6e7f085650-kube-api-access-pvxpc" (OuterVolumeSpecName: "kube-api-access-pvxpc") pod "fc444786-9036-4e36-9f80-5e6e7f085650" (UID: "fc444786-9036-4e36-9f80-5e6e7f085650"). InnerVolumeSpecName "kube-api-access-pvxpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:19:24.447330 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:24.447302 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvxpc\" (UniqueName: \"kubernetes.io/projected/fc444786-9036-4e36-9f80-5e6e7f085650-kube-api-access-pvxpc\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:25.009966 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.009927 2559 generic.go:356] "Generic (PLEG): container finished" podID="fc444786-9036-4e36-9f80-5e6e7f085650" containerID="a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667" exitCode=0 Jul 07 02:19:25.010441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.009985 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" Jul 07 02:19:25.010441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.010011 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" event={"ID":"fc444786-9036-4e36-9f80-5e6e7f085650","Type":"ContainerDied","Data":"a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667"} Jul 07 02:19:25.010441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.010080 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lc22c" event={"ID":"fc444786-9036-4e36-9f80-5e6e7f085650","Type":"ContainerDied","Data":"bec598b947dfd6c6c990762ec2ccfcee8cc33ee4586db27a1e204781f66d53ac"} Jul 07 02:19:25.010441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.010100 2559 scope.go:117] "RemoveContainer" containerID="a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667" Jul 07 02:19:25.019567 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.019548 2559 scope.go:117] "RemoveContainer" containerID="a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667" Jul 07 02:19:25.019848 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:19:25.019825 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667\": container with ID starting with a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667 not found: ID does not exist" containerID="a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667" Jul 07 02:19:25.019940 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.019853 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667"} err="failed to get container status \"a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667\": rpc error: code = NotFound desc = could not find container \"a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667\": container with ID starting with a0765683906f0b3101e005fa63f20c6e0ef4f9c853cbd3a750a2689ecfc64667 not found: ID does not exist" Jul 07 02:19:25.030440 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.030418 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lc22c"] Jul 07 02:19:25.034014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.033992 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lc22c"] Jul 07 02:19:25.818773 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:25.818739 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc444786-9036-4e36-9f80-5e6e7f085650" path="/var/lib/kubelet/pods/fc444786-9036-4e36-9f80-5e6e7f085650/volumes" Jul 07 02:19:31.621522 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:31.621492 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:31.626978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:31.621619 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:31.649694 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:31.649670 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:32.066879 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:32.066848 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ptn4f" Jul 07 02:19:36.344266 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.344229 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq"] Jul 07 02:19:36.344739 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.344701 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc444786-9036-4e36-9f80-5e6e7f085650" containerName="registry-server" Jul 07 02:19:36.344739 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.344719 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc444786-9036-4e36-9f80-5e6e7f085650" containerName="registry-server" Jul 07 02:19:36.344849 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.344841 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc444786-9036-4e36-9f80-5e6e7f085650" containerName="registry-server" Jul 07 02:19:36.347952 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.347932 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.350214 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.350191 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2gnbx\"" Jul 07 02:19:36.357190 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.357146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.357317 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.357270 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.357317 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.357301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2wvm\" (UniqueName: \"kubernetes.io/projected/61bc2748-4d91-4d73-a339-e60011fe4670-kube-api-access-w2wvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.357419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.357305 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq"] Jul 07 02:19:36.457894 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.457849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.457894 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.457890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2wvm\" (UniqueName: \"kubernetes.io/projected/61bc2748-4d91-4d73-a339-e60011fe4670-kube-api-access-w2wvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.458185 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.458078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.458342 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.458321 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.458415 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.458397 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.466339 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.466316 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2wvm\" (UniqueName: \"kubernetes.io/projected/61bc2748-4d91-4d73-a339-e60011fe4670-kube-api-access-w2wvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.656817 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.656786 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:36.786178 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:36.786137 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq"] Jul 07 02:19:36.787760 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:36.787729 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61bc2748_4d91_4d73_a339_e60011fe4670.slice/crio-30c308cadfc4732443959a72c4e4e0cc3117a246d8ee044c3793aad29a8e5bbf WatchSource:0}: Error finding container 30c308cadfc4732443959a72c4e4e0cc3117a246d8ee044c3793aad29a8e5bbf: Status 404 returned error can't find the container with id 30c308cadfc4732443959a72c4e4e0cc3117a246d8ee044c3793aad29a8e5bbf Jul 07 02:19:37.069222 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.069104 2559 generic.go:356] "Generic (PLEG): container finished" podID="61bc2748-4d91-4d73-a339-e60011fe4670" containerID="fbaaf1daf982b1249c242a1419efe49ed36d0ba975e75d1c7fd97c2ab218c15a" exitCode=0 Jul 07 02:19:37.069222 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.069179 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" event={"ID":"61bc2748-4d91-4d73-a339-e60011fe4670","Type":"ContainerDied","Data":"fbaaf1daf982b1249c242a1419efe49ed36d0ba975e75d1c7fd97c2ab218c15a"} Jul 07 02:19:37.069222 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.069214 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" event={"ID":"61bc2748-4d91-4d73-a339-e60011fe4670","Type":"ContainerStarted","Data":"30c308cadfc4732443959a72c4e4e0cc3117a246d8ee044c3793aad29a8e5bbf"} Jul 07 02:19:37.143554 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.143520 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn"] Jul 07 02:19:37.147113 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.147097 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.160447 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.160423 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn"] Jul 07 02:19:37.162060 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.162039 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x456b\" (UniqueName: \"kubernetes.io/projected/8461e185-e859-40de-93a8-a136086d7639-kube-api-access-x456b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.162216 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.162108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.162216 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.162182 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.263080 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.263048 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x456b\" (UniqueName: \"kubernetes.io/projected/8461e185-e859-40de-93a8-a136086d7639-kube-api-access-x456b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.263317 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.263141 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.263317 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.263196 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.263522 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.263504 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.263587 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.263565 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.272391 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.272368 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x456b\" (UniqueName: \"kubernetes.io/projected/8461e185-e859-40de-93a8-a136086d7639-kube-api-access-x456b\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.456624 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.456598 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:37.590767 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.590742 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn"] Jul 07 02:19:37.702326 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:37.702282 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8461e185_e859_40de_93a8_a136086d7639.slice/crio-4101c73aa6438c2b4fbdedaac72115195ec7788d45c9d7d4d8775ad5ab283eda WatchSource:0}: Error finding container 4101c73aa6438c2b4fbdedaac72115195ec7788d45c9d7d4d8775ad5ab283eda: Status 404 returned error can't find the container with id 4101c73aa6438c2b4fbdedaac72115195ec7788d45c9d7d4d8775ad5ab283eda Jul 07 02:19:37.938885 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.938856 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw"] Jul 07 02:19:37.942464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.942450 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:37.953756 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.953729 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw"] Jul 07 02:19:37.969042 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.968976 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:37.969042 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.969009 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw8r2\" (UniqueName: \"kubernetes.io/projected/49d0c21e-e32f-41fc-b548-6143959ee966-kube-api-access-qw8r2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:37.969042 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:37.969041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.070260 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.070230 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.070397 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.070268 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw8r2\" (UniqueName: \"kubernetes.io/projected/49d0c21e-e32f-41fc-b548-6143959ee966-kube-api-access-qw8r2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.070397 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.070297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.070788 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.070763 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.070898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.070803 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.076249 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.076223 2559 generic.go:356] "Generic (PLEG): container finished" podID="61bc2748-4d91-4d73-a339-e60011fe4670" containerID="ba22995e196171c8da67d3f1b312bc8a902e8985d74a87b9f95105d85948f104" exitCode=0 Jul 07 02:19:38.076357 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.076262 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" event={"ID":"61bc2748-4d91-4d73-a339-e60011fe4670","Type":"ContainerDied","Data":"ba22995e196171c8da67d3f1b312bc8a902e8985d74a87b9f95105d85948f104"} Jul 07 02:19:38.077953 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.077905 2559 generic.go:356] "Generic (PLEG): container finished" podID="8461e185-e859-40de-93a8-a136086d7639" containerID="3af5d2c3c1f639ed1a5335bedd210383e9de0f98e5ed8de0380a85b7c8d5c182" exitCode=0 Jul 07 02:19:38.078017 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.077950 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" event={"ID":"8461e185-e859-40de-93a8-a136086d7639","Type":"ContainerDied","Data":"3af5d2c3c1f639ed1a5335bedd210383e9de0f98e5ed8de0380a85b7c8d5c182"} Jul 07 02:19:38.078017 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.078010 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" event={"ID":"8461e185-e859-40de-93a8-a136086d7639","Type":"ContainerStarted","Data":"4101c73aa6438c2b4fbdedaac72115195ec7788d45c9d7d4d8775ad5ab283eda"} Jul 07 02:19:38.079542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.079523 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw8r2\" (UniqueName: \"kubernetes.io/projected/49d0c21e-e32f-41fc-b548-6143959ee966-kube-api-access-qw8r2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.251226 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.251127 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:38.344624 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.344593 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2"] Jul 07 02:19:38.350310 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.350281 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.354800 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.354774 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2"] Jul 07 02:19:38.372875 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.372849 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlggd\" (UniqueName: \"kubernetes.io/projected/90671ba1-dec9-42e4-8c3f-d24a253b17cc-kube-api-access-qlggd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.373014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.372882 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.373014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.372925 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.384555 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.384536 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw"] Jul 07 02:19:38.387056 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:38.387032 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d0c21e_e32f_41fc_b548_6143959ee966.slice/crio-3a1afa4ac4f8e61c07a0e550b9f0ac775b9e32d8ad91beef935c55f3b8d3d6ba WatchSource:0}: Error finding container 3a1afa4ac4f8e61c07a0e550b9f0ac775b9e32d8ad91beef935c55f3b8d3d6ba: Status 404 returned error can't find the container with id 3a1afa4ac4f8e61c07a0e550b9f0ac775b9e32d8ad91beef935c55f3b8d3d6ba Jul 07 02:19:38.474343 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.474310 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlggd\" (UniqueName: \"kubernetes.io/projected/90671ba1-dec9-42e4-8c3f-d24a253b17cc-kube-api-access-qlggd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.474686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.474347 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.474686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.474391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.474777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.474763 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.474812 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.474778 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.482818 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.482789 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlggd\" (UniqueName: \"kubernetes.io/projected/90671ba1-dec9-42e4-8c3f-d24a253b17cc-kube-api-access-qlggd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.661642 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.661617 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:38.794797 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:38.794771 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2"] Jul 07 02:19:38.826560 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:38.826528 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90671ba1_dec9_42e4_8c3f_d24a253b17cc.slice/crio-4f1898fe1f789b20b07ecdc59dba79ed8986ea06149d46378feaf39accf245d2 WatchSource:0}: Error finding container 4f1898fe1f789b20b07ecdc59dba79ed8986ea06149d46378feaf39accf245d2: Status 404 returned error can't find the container with id 4f1898fe1f789b20b07ecdc59dba79ed8986ea06149d46378feaf39accf245d2 Jul 07 02:19:39.084029 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.083993 2559 generic.go:356] "Generic (PLEG): container finished" podID="49d0c21e-e32f-41fc-b548-6143959ee966" containerID="b62b3054929a64ece5634b8fbc828117c5a57818e2db14a357d16b52da2023a5" exitCode=0 Jul 07 02:19:39.084190 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.084073 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" event={"ID":"49d0c21e-e32f-41fc-b548-6143959ee966","Type":"ContainerDied","Data":"b62b3054929a64ece5634b8fbc828117c5a57818e2db14a357d16b52da2023a5"} Jul 07 02:19:39.084190 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.084170 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" event={"ID":"49d0c21e-e32f-41fc-b548-6143959ee966","Type":"ContainerStarted","Data":"3a1afa4ac4f8e61c07a0e550b9f0ac775b9e32d8ad91beef935c55f3b8d3d6ba"} Jul 07 02:19:39.086392 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.086372 2559 generic.go:356] "Generic (PLEG): container finished" podID="61bc2748-4d91-4d73-a339-e60011fe4670" containerID="50d2b98d12983033ab494e38318487feaf710068dbc316fec925560f63199f63" exitCode=0 Jul 07 02:19:39.086519 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.086436 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" event={"ID":"61bc2748-4d91-4d73-a339-e60011fe4670","Type":"ContainerDied","Data":"50d2b98d12983033ab494e38318487feaf710068dbc316fec925560f63199f63"} Jul 07 02:19:39.088247 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.088226 2559 generic.go:356] "Generic (PLEG): container finished" podID="8461e185-e859-40de-93a8-a136086d7639" containerID="bea16c569e832e925efa5e614639397b1afa156be9020637b7f17ccb1d72e916" exitCode=0 Jul 07 02:19:39.088347 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.088332 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" event={"ID":"8461e185-e859-40de-93a8-a136086d7639","Type":"ContainerDied","Data":"bea16c569e832e925efa5e614639397b1afa156be9020637b7f17ccb1d72e916"} Jul 07 02:19:39.089868 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.089845 2559 generic.go:356] "Generic (PLEG): container finished" podID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerID="27aff8c12fc94601091291f6cfb9f7f75b7f000954a7ad3491cfbe7aeed01c60" exitCode=0 Jul 07 02:19:39.089966 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.089909 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" event={"ID":"90671ba1-dec9-42e4-8c3f-d24a253b17cc","Type":"ContainerDied","Data":"27aff8c12fc94601091291f6cfb9f7f75b7f000954a7ad3491cfbe7aeed01c60"} Jul 07 02:19:39.089966 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:39.089949 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" event={"ID":"90671ba1-dec9-42e4-8c3f-d24a253b17cc","Type":"ContainerStarted","Data":"4f1898fe1f789b20b07ecdc59dba79ed8986ea06149d46378feaf39accf245d2"} Jul 07 02:19:40.096788 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.096748 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" event={"ID":"49d0c21e-e32f-41fc-b548-6143959ee966","Type":"ContainerStarted","Data":"836ae1bd1e561909472d8c2ffb0ee7ac2351e18e6525486a627df16d7fba03eb"} Jul 07 02:19:40.098974 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.098951 2559 generic.go:356] "Generic (PLEG): container finished" podID="8461e185-e859-40de-93a8-a136086d7639" containerID="ff3ec80a3f03fe7be40c9721b0cbc995545cfbfbd93bf4da3e0dc70dab51ee03" exitCode=0 Jul 07 02:19:40.099064 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.099010 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" event={"ID":"8461e185-e859-40de-93a8-a136086d7639","Type":"ContainerDied","Data":"ff3ec80a3f03fe7be40c9721b0cbc995545cfbfbd93bf4da3e0dc70dab51ee03"} Jul 07 02:19:40.100967 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.100943 2559 generic.go:356] "Generic (PLEG): container finished" podID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerID="d11d9c957ad8a4abf262526d74d8f2356defdee02b99bfe9f3f2b8557974bb7c" exitCode=0 Jul 07 02:19:40.101132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.100999 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" event={"ID":"90671ba1-dec9-42e4-8c3f-d24a253b17cc","Type":"ContainerDied","Data":"d11d9c957ad8a4abf262526d74d8f2356defdee02b99bfe9f3f2b8557974bb7c"} Jul 07 02:19:40.227612 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.227592 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:40.292176 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.292134 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2wvm\" (UniqueName: \"kubernetes.io/projected/61bc2748-4d91-4d73-a339-e60011fe4670-kube-api-access-w2wvm\") pod \"61bc2748-4d91-4d73-a339-e60011fe4670\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " Jul 07 02:19:40.292305 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.292284 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-util\") pod \"61bc2748-4d91-4d73-a339-e60011fe4670\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " Jul 07 02:19:40.292442 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.292315 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-bundle\") pod \"61bc2748-4d91-4d73-a339-e60011fe4670\" (UID: \"61bc2748-4d91-4d73-a339-e60011fe4670\") " Jul 07 02:19:40.292831 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.292799 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-bundle" (OuterVolumeSpecName: "bundle") pod "61bc2748-4d91-4d73-a339-e60011fe4670" (UID: "61bc2748-4d91-4d73-a339-e60011fe4670"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:40.294124 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.294100 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61bc2748-4d91-4d73-a339-e60011fe4670-kube-api-access-w2wvm" (OuterVolumeSpecName: "kube-api-access-w2wvm") pod "61bc2748-4d91-4d73-a339-e60011fe4670" (UID: "61bc2748-4d91-4d73-a339-e60011fe4670"). InnerVolumeSpecName "kube-api-access-w2wvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:19:40.297129 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.297110 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-util" (OuterVolumeSpecName: "util") pod "61bc2748-4d91-4d73-a339-e60011fe4670" (UID: "61bc2748-4d91-4d73-a339-e60011fe4670"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:40.393898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.393818 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2wvm\" (UniqueName: \"kubernetes.io/projected/61bc2748-4d91-4d73-a339-e60011fe4670-kube-api-access-w2wvm\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:40.393898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.393842 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-util\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:40.393898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:40.393855 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61bc2748-4d91-4d73-a339-e60011fe4670-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:41.108207 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.108176 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" Jul 07 02:19:41.108207 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.108196 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq" event={"ID":"61bc2748-4d91-4d73-a339-e60011fe4670","Type":"ContainerDied","Data":"30c308cadfc4732443959a72c4e4e0cc3117a246d8ee044c3793aad29a8e5bbf"} Jul 07 02:19:41.108680 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.108249 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30c308cadfc4732443959a72c4e4e0cc3117a246d8ee044c3793aad29a8e5bbf" Jul 07 02:19:41.110617 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.110522 2559 generic.go:356] "Generic (PLEG): container finished" podID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerID="d9d66eb8391779efadbd904aae946ade4c17d0a9b02de9ad749ff7f4de97a974" exitCode=0 Jul 07 02:19:41.110617 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.110566 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" event={"ID":"90671ba1-dec9-42e4-8c3f-d24a253b17cc","Type":"ContainerDied","Data":"d9d66eb8391779efadbd904aae946ade4c17d0a9b02de9ad749ff7f4de97a974"} Jul 07 02:19:41.112759 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.112735 2559 generic.go:356] "Generic (PLEG): container finished" podID="49d0c21e-e32f-41fc-b548-6143959ee966" containerID="836ae1bd1e561909472d8c2ffb0ee7ac2351e18e6525486a627df16d7fba03eb" exitCode=0 Jul 07 02:19:41.112871 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.112820 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" event={"ID":"49d0c21e-e32f-41fc-b548-6143959ee966","Type":"ContainerDied","Data":"836ae1bd1e561909472d8c2ffb0ee7ac2351e18e6525486a627df16d7fba03eb"} Jul 07 02:19:41.248489 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.248461 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:41.301669 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.301639 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-bundle\") pod \"8461e185-e859-40de-93a8-a136086d7639\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " Jul 07 02:19:41.301838 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.301714 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x456b\" (UniqueName: \"kubernetes.io/projected/8461e185-e859-40de-93a8-a136086d7639-kube-api-access-x456b\") pod \"8461e185-e859-40de-93a8-a136086d7639\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " Jul 07 02:19:41.301838 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.301736 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-util\") pod \"8461e185-e859-40de-93a8-a136086d7639\" (UID: \"8461e185-e859-40de-93a8-a136086d7639\") " Jul 07 02:19:41.302272 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.302244 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-bundle" (OuterVolumeSpecName: "bundle") pod "8461e185-e859-40de-93a8-a136086d7639" (UID: "8461e185-e859-40de-93a8-a136086d7639"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:41.303790 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.303770 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8461e185-e859-40de-93a8-a136086d7639-kube-api-access-x456b" (OuterVolumeSpecName: "kube-api-access-x456b") pod "8461e185-e859-40de-93a8-a136086d7639" (UID: "8461e185-e859-40de-93a8-a136086d7639"). InnerVolumeSpecName "kube-api-access-x456b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:19:41.306558 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.306532 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-util" (OuterVolumeSpecName: "util") pod "8461e185-e859-40de-93a8-a136086d7639" (UID: "8461e185-e859-40de-93a8-a136086d7639"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:41.402417 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.402382 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x456b\" (UniqueName: \"kubernetes.io/projected/8461e185-e859-40de-93a8-a136086d7639-kube-api-access-x456b\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:41.402417 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.402417 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-util\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:41.402597 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:41.402433 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8461e185-e859-40de-93a8-a136086d7639-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:42.120089 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.120002 2559 generic.go:356] "Generic (PLEG): container finished" podID="49d0c21e-e32f-41fc-b548-6143959ee966" containerID="ef8188e17733c9ab46792d7cba6aea1b0d17553bffbc561971ab20654d71d735" exitCode=0 Jul 07 02:19:42.120509 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.120085 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" event={"ID":"49d0c21e-e32f-41fc-b548-6143959ee966","Type":"ContainerDied","Data":"ef8188e17733c9ab46792d7cba6aea1b0d17553bffbc561971ab20654d71d735"} Jul 07 02:19:42.122031 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.122011 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" Jul 07 02:19:42.122136 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.122041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn" event={"ID":"8461e185-e859-40de-93a8-a136086d7639","Type":"ContainerDied","Data":"4101c73aa6438c2b4fbdedaac72115195ec7788d45c9d7d4d8775ad5ab283eda"} Jul 07 02:19:42.122136 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.122071 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4101c73aa6438c2b4fbdedaac72115195ec7788d45c9d7d4d8775ad5ab283eda" Jul 07 02:19:42.255003 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.254978 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:42.310921 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.310887 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-util\") pod \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " Jul 07 02:19:42.311091 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.310958 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-bundle\") pod \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " Jul 07 02:19:42.311091 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.310992 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlggd\" (UniqueName: \"kubernetes.io/projected/90671ba1-dec9-42e4-8c3f-d24a253b17cc-kube-api-access-qlggd\") pod \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\" (UID: \"90671ba1-dec9-42e4-8c3f-d24a253b17cc\") " Jul 07 02:19:42.311496 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.311472 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-bundle" (OuterVolumeSpecName: "bundle") pod "90671ba1-dec9-42e4-8c3f-d24a253b17cc" (UID: "90671ba1-dec9-42e4-8c3f-d24a253b17cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:42.313215 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.313191 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90671ba1-dec9-42e4-8c3f-d24a253b17cc-kube-api-access-qlggd" (OuterVolumeSpecName: "kube-api-access-qlggd") pod "90671ba1-dec9-42e4-8c3f-d24a253b17cc" (UID: "90671ba1-dec9-42e4-8c3f-d24a253b17cc"). InnerVolumeSpecName "kube-api-access-qlggd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:19:42.316713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.316682 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-util" (OuterVolumeSpecName: "util") pod "90671ba1-dec9-42e4-8c3f-d24a253b17cc" (UID: "90671ba1-dec9-42e4-8c3f-d24a253b17cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:42.412471 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.412438 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:42.412471 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.412465 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlggd\" (UniqueName: \"kubernetes.io/projected/90671ba1-dec9-42e4-8c3f-d24a253b17cc-kube-api-access-qlggd\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:42.412471 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:42.412475 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90671ba1-dec9-42e4-8c3f-d24a253b17cc-util\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:43.129084 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.129059 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" Jul 07 02:19:43.129084 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.129068 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2" event={"ID":"90671ba1-dec9-42e4-8c3f-d24a253b17cc","Type":"ContainerDied","Data":"4f1898fe1f789b20b07ecdc59dba79ed8986ea06149d46378feaf39accf245d2"} Jul 07 02:19:43.129528 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.129121 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f1898fe1f789b20b07ecdc59dba79ed8986ea06149d46378feaf39accf245d2" Jul 07 02:19:43.257519 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.257495 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:43.321555 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.321522 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-util\") pod \"49d0c21e-e32f-41fc-b548-6143959ee966\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " Jul 07 02:19:43.321713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.321618 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-bundle\") pod \"49d0c21e-e32f-41fc-b548-6143959ee966\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " Jul 07 02:19:43.321775 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.321754 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw8r2\" (UniqueName: \"kubernetes.io/projected/49d0c21e-e32f-41fc-b548-6143959ee966-kube-api-access-qw8r2\") pod \"49d0c21e-e32f-41fc-b548-6143959ee966\" (UID: \"49d0c21e-e32f-41fc-b548-6143959ee966\") " Jul 07 02:19:43.322177 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.322127 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-bundle" (OuterVolumeSpecName: "bundle") pod "49d0c21e-e32f-41fc-b548-6143959ee966" (UID: "49d0c21e-e32f-41fc-b548-6143959ee966"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:43.323776 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.323750 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d0c21e-e32f-41fc-b548-6143959ee966-kube-api-access-qw8r2" (OuterVolumeSpecName: "kube-api-access-qw8r2") pod "49d0c21e-e32f-41fc-b548-6143959ee966" (UID: "49d0c21e-e32f-41fc-b548-6143959ee966"). InnerVolumeSpecName "kube-api-access-qw8r2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:19:43.327119 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.327098 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-util" (OuterVolumeSpecName: "util") pod "49d0c21e-e32f-41fc-b548-6143959ee966" (UID: "49d0c21e-e32f-41fc-b548-6143959ee966"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:19:43.423566 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.423531 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:43.423566 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.423566 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw8r2\" (UniqueName: \"kubernetes.io/projected/49d0c21e-e32f-41fc-b548-6143959ee966-kube-api-access-qw8r2\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:43.423759 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:43.423578 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49d0c21e-e32f-41fc-b548-6143959ee966-util\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:19:44.135832 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:44.135743 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" event={"ID":"49d0c21e-e32f-41fc-b548-6143959ee966","Type":"ContainerDied","Data":"3a1afa4ac4f8e61c07a0e550b9f0ac775b9e32d8ad91beef935c55f3b8d3d6ba"} Jul 07 02:19:44.135832 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:44.135765 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw" Jul 07 02:19:44.135832 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:44.135800 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a1afa4ac4f8e61c07a0e550b9f0ac775b9e32d8ad91beef935c55f3b8d3d6ba" Jul 07 02:19:53.819281 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819253 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl"] Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819548 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8461e185-e859-40de-93a8-a136086d7639" containerName="extract" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819560 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8461e185-e859-40de-93a8-a136086d7639" containerName="extract" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819568 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d0c21e-e32f-41fc-b548-6143959ee966" containerName="pull" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819573 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d0c21e-e32f-41fc-b548-6143959ee966" containerName="pull" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819582 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerName="util" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819588 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerName="util" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819596 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerName="extract" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819601 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerName="extract" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819609 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerName="pull" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819614 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerName="pull" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819622 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d0c21e-e32f-41fc-b548-6143959ee966" containerName="util" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819626 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d0c21e-e32f-41fc-b548-6143959ee966" containerName="util" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819633 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d0c21e-e32f-41fc-b548-6143959ee966" containerName="extract" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819638 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d0c21e-e32f-41fc-b548-6143959ee966" containerName="extract" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819650 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8461e185-e859-40de-93a8-a136086d7639" containerName="util" Jul 07 02:19:53.819655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819655 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8461e185-e859-40de-93a8-a136086d7639" containerName="util" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819664 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bc2748-4d91-4d73-a339-e60011fe4670" containerName="util" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819669 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bc2748-4d91-4d73-a339-e60011fe4670" containerName="util" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819675 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8461e185-e859-40de-93a8-a136086d7639" containerName="pull" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819679 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8461e185-e859-40de-93a8-a136086d7639" containerName="pull" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819686 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bc2748-4d91-4d73-a339-e60011fe4670" containerName="pull" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819691 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bc2748-4d91-4d73-a339-e60011fe4670" containerName="pull" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819696 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61bc2748-4d91-4d73-a339-e60011fe4670" containerName="extract" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819701 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="61bc2748-4d91-4d73-a339-e60011fe4670" containerName="extract" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819749 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d0c21e-e32f-41fc-b548-6143959ee966" containerName="extract" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819758 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="61bc2748-4d91-4d73-a339-e60011fe4670" containerName="extract" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819764 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="90671ba1-dec9-42e4-8c3f-d24a253b17cc" containerName="extract" Jul 07 02:19:53.820109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.819770 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8461e185-e859-40de-93a8-a136086d7639" containerName="extract" Jul 07 02:19:53.822761 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.822742 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" Jul 07 02:19:53.825459 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.825435 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kr5rw\"" Jul 07 02:19:53.826658 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.826633 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl"] Jul 07 02:19:53.914567 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:53.914530 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl68b\" (UniqueName: \"kubernetes.io/projected/9a87265c-48a1-45a4-a73b-236653a87e37-kube-api-access-hl68b\") pod \"authorino-operator-75cc8d5cb7-wrrdl\" (UID: \"9a87265c-48a1-45a4-a73b-236653a87e37\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" Jul 07 02:19:54.015975 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:54.015941 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl68b\" (UniqueName: \"kubernetes.io/projected/9a87265c-48a1-45a4-a73b-236653a87e37-kube-api-access-hl68b\") pod \"authorino-operator-75cc8d5cb7-wrrdl\" (UID: \"9a87265c-48a1-45a4-a73b-236653a87e37\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" Jul 07 02:19:54.036048 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:54.036014 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl68b\" (UniqueName: \"kubernetes.io/projected/9a87265c-48a1-45a4-a73b-236653a87e37-kube-api-access-hl68b\") pod \"authorino-operator-75cc8d5cb7-wrrdl\" (UID: \"9a87265c-48a1-45a4-a73b-236653a87e37\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" Jul 07 02:19:54.133681 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:54.133597 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" Jul 07 02:19:54.262728 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:54.262703 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl"] Jul 07 02:19:54.264616 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:54.264589 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a87265c_48a1_45a4_a73b_236653a87e37.slice/crio-9dfdcb6d2b9e00622f5748d1b6311a698b199d33db9f181aebbc914b7e913b2e WatchSource:0}: Error finding container 9dfdcb6d2b9e00622f5748d1b6311a698b199d33db9f181aebbc914b7e913b2e: Status 404 returned error can't find the container with id 9dfdcb6d2b9e00622f5748d1b6311a698b199d33db9f181aebbc914b7e913b2e Jul 07 02:19:54.266691 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:54.266675 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 02:19:55.194739 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:55.194702 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" event={"ID":"9a87265c-48a1-45a4-a73b-236653a87e37","Type":"ContainerStarted","Data":"9dfdcb6d2b9e00622f5748d1b6311a698b199d33db9f181aebbc914b7e913b2e"} Jul 07 02:19:57.045717 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.045684 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj"] Jul 07 02:19:57.053798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.053780 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:19:57.056208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.056185 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5rxcx\"" Jul 07 02:19:57.061618 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.061594 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj"] Jul 07 02:19:57.146281 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.146250 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5jgx\" (UniqueName: \"kubernetes.io/projected/b690989d-9d8f-44a1-acfe-a4f528760dfa-kube-api-access-g5jgx\") pod \"limitador-operator-controller-manager-859d7b78bd-q2hfj\" (UID: \"b690989d-9d8f-44a1-acfe-a4f528760dfa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:19:57.207163 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.207114 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" event={"ID":"9a87265c-48a1-45a4-a73b-236653a87e37","Type":"ContainerStarted","Data":"43dff32286a5d37e1322787f0c130d187081397f1b305d10297523a9bed1d941"} Jul 07 02:19:57.207323 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.207306 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" Jul 07 02:19:57.226134 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.226088 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" podStartSLOduration=2.141467525 podStartE2EDuration="4.226074214s" totalImagesPullingTime="2.084606689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:19:57.225792596 +0000 UTC m=+613.976214250" watchObservedRunningTime="2026-07-07 02:19:57.226074214 +0000 UTC m=+613.976495908" Jul 07 02:19:57.247580 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.247551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5jgx\" (UniqueName: \"kubernetes.io/projected/b690989d-9d8f-44a1-acfe-a4f528760dfa-kube-api-access-g5jgx\") pod \"limitador-operator-controller-manager-859d7b78bd-q2hfj\" (UID: \"b690989d-9d8f-44a1-acfe-a4f528760dfa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:19:57.260276 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.260247 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5jgx\" (UniqueName: \"kubernetes.io/projected/b690989d-9d8f-44a1-acfe-a4f528760dfa-kube-api-access-g5jgx\") pod \"limitador-operator-controller-manager-859d7b78bd-q2hfj\" (UID: \"b690989d-9d8f-44a1-acfe-a4f528760dfa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:19:57.372549 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.372465 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:19:57.508245 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:57.508223 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj"] Jul 07 02:19:57.510968 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:19:57.510942 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb690989d_9d8f_44a1_acfe_a4f528760dfa.slice/crio-4c5b561901d943edb0b394d5728eaa1dbd482be856ba46cd462de78d49e8cd24 WatchSource:0}: Error finding container 4c5b561901d943edb0b394d5728eaa1dbd482be856ba46cd462de78d49e8cd24: Status 404 returned error can't find the container with id 4c5b561901d943edb0b394d5728eaa1dbd482be856ba46cd462de78d49e8cd24 Jul 07 02:19:58.214708 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:19:58.214677 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" event={"ID":"b690989d-9d8f-44a1-acfe-a4f528760dfa","Type":"ContainerStarted","Data":"4c5b561901d943edb0b394d5728eaa1dbd482be856ba46cd462de78d49e8cd24"} Jul 07 02:20:00.227621 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:00.227586 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" event={"ID":"b690989d-9d8f-44a1-acfe-a4f528760dfa","Type":"ContainerStarted","Data":"89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054"} Jul 07 02:20:00.228025 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:00.227720 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:00.254652 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:00.254605 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" podStartSLOduration=1.469213664 podStartE2EDuration="3.254590641s" totalImagesPullingTime="1.785376977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:00.252673178 +0000 UTC m=+617.003094832" watchObservedRunningTime="2026-07-07 02:20:00.254590641 +0000 UTC m=+617.005012293" Jul 07 02:20:08.216995 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:08.216911 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wrrdl" Jul 07 02:20:11.235722 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:11.235689 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:12.996773 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:12.996737 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2"] Jul 07 02:20:13.001683 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.001664 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.003900 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.003880 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 02:20:13.004033 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.004015 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 02:20:13.004269 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.004245 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2gnbx\"" Jul 07 02:20:13.011088 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.011064 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2"] Jul 07 02:20:13.086481 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.086446 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/419c705b-aaa6-4472-a85a-57d2ce84f40f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.086652 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.086511 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/419c705b-aaa6-4472-a85a-57d2ce84f40f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.086652 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.086553 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvckt\" (UniqueName: \"kubernetes.io/projected/419c705b-aaa6-4472-a85a-57d2ce84f40f-kube-api-access-qvckt\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.187626 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.187594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/419c705b-aaa6-4472-a85a-57d2ce84f40f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.187804 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.187648 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvckt\" (UniqueName: \"kubernetes.io/projected/419c705b-aaa6-4472-a85a-57d2ce84f40f-kube-api-access-qvckt\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.187804 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.187715 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/419c705b-aaa6-4472-a85a-57d2ce84f40f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.188445 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.188420 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/419c705b-aaa6-4472-a85a-57d2ce84f40f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.190171 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.190135 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/419c705b-aaa6-4472-a85a-57d2ce84f40f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.195383 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.195364 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvckt\" (UniqueName: \"kubernetes.io/projected/419c705b-aaa6-4472-a85a-57d2ce84f40f-kube-api-access-qvckt\") pod \"kuadrant-console-plugin-66b5774985-z4nd2\" (UID: \"419c705b-aaa6-4472-a85a-57d2ce84f40f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.312291 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.312207 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" Jul 07 02:20:13.442992 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:13.442962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2"] Jul 07 02:20:13.444125 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:20:13.444104 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod419c705b_aaa6_4472_a85a_57d2ce84f40f.slice/crio-395252ef5caac3f2e5db85aed6d551dba7ed73eda3087297a769adfaa92084f3 WatchSource:0}: Error finding container 395252ef5caac3f2e5db85aed6d551dba7ed73eda3087297a769adfaa92084f3: Status 404 returned error can't find the container with id 395252ef5caac3f2e5db85aed6d551dba7ed73eda3087297a769adfaa92084f3 Jul 07 02:20:14.108093 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:14.108062 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9fcbf6949-x7q8x"] Jul 07 02:20:14.300272 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:14.300236 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" event={"ID":"419c705b-aaa6-4472-a85a-57d2ce84f40f","Type":"ContainerStarted","Data":"395252ef5caac3f2e5db85aed6d551dba7ed73eda3087297a769adfaa92084f3"} Jul 07 02:20:23.884753 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:23.884719 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj"] Jul 07 02:20:23.887245 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:23.884943 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" containerName="manager" containerID="cri-o://89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054" gracePeriod=2 Jul 07 02:20:23.888015 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:20:23.887980 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q2hfj\" is forbidden: User \"system:node:ip-10-0-141-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-30.ec2.internal' and this object" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:23.890897 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:23.890869 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj"] Jul 07 02:20:24.147193 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.147118 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:24.149586 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:20:24.149547 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q2hfj\" is forbidden: User \"system:node:ip-10-0-141-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-30.ec2.internal' and this object" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:24.192047 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.192012 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5jgx\" (UniqueName: \"kubernetes.io/projected/b690989d-9d8f-44a1-acfe-a4f528760dfa-kube-api-access-g5jgx\") pod \"b690989d-9d8f-44a1-acfe-a4f528760dfa\" (UID: \"b690989d-9d8f-44a1-acfe-a4f528760dfa\") " Jul 07 02:20:24.194427 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.194391 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b690989d-9d8f-44a1-acfe-a4f528760dfa-kube-api-access-g5jgx" (OuterVolumeSpecName: "kube-api-access-g5jgx") pod "b690989d-9d8f-44a1-acfe-a4f528760dfa" (UID: "b690989d-9d8f-44a1-acfe-a4f528760dfa"). InnerVolumeSpecName "kube-api-access-g5jgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:20:24.292714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.292680 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5jgx\" (UniqueName: \"kubernetes.io/projected/b690989d-9d8f-44a1-acfe-a4f528760dfa-kube-api-access-g5jgx\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:24.359000 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.358964 2559 generic.go:356] "Generic (PLEG): container finished" podID="b690989d-9d8f-44a1-acfe-a4f528760dfa" containerID="89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054" exitCode=0 Jul 07 02:20:24.359208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.359017 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:24.359208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.359078 2559 scope.go:117] "RemoveContainer" containerID="89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054" Jul 07 02:20:24.361587 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:20:24.361555 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q2hfj\" is forbidden: User \"system:node:ip-10-0-141-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-30.ec2.internal' and this object" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:24.370645 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.370625 2559 scope.go:117] "RemoveContainer" containerID="89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054" Jul 07 02:20:24.370976 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:20:24.370951 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054\": container with ID starting with 89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054 not found: ID does not exist" containerID="89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054" Jul 07 02:20:24.371044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:24.370987 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054"} err="failed to get container status \"89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054\": rpc error: code = NotFound desc = could not find container \"89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054\": container with ID starting with 89ddfaf6d65ecbec98ab43a762c9916c517ca67ca1392046b6226b6a2fd1e054 not found: ID does not exist" Jul 07 02:20:24.371513 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:20:24.371483 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q2hfj\" is forbidden: User \"system:node:ip-10-0-141-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-30.ec2.internal' and this object" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q2hfj" Jul 07 02:20:25.823109 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:25.823077 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" path="/var/lib/kubelet/pods/b690989d-9d8f-44a1-acfe-a4f528760dfa/volumes" Jul 07 02:20:39.128673 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.128629 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-9fcbf6949-x7q8x" podUID="5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" containerName="console" containerID="cri-o://7cae48c63d920208b66c12479ceae8f68dd9d67e942b3a5a42d0f032e80c1df0" gracePeriod=15 Jul 07 02:20:39.461275 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.461234 2559 generic.go:356] "Generic (PLEG): container finished" podID="5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" containerID="7cae48c63d920208b66c12479ceae8f68dd9d67e942b3a5a42d0f032e80c1df0" exitCode=0 Jul 07 02:20:39.461429 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.461302 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fcbf6949-x7q8x" event={"ID":"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb","Type":"ContainerDied","Data":"7cae48c63d920208b66c12479ceae8f68dd9d67e942b3a5a42d0f032e80c1df0"} Jul 07 02:20:39.574570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.574547 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:20:39.637483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.637456 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8tq7\" (UniqueName: \"kubernetes.io/projected/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-kube-api-access-q8tq7\") pod \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " Jul 07 02:20:39.637655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.637513 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-oauth-config\") pod \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " Jul 07 02:20:39.637655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.637553 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-oauth-serving-cert\") pod \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " Jul 07 02:20:39.637655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.637586 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-trusted-ca-bundle\") pod \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " Jul 07 02:20:39.637655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.637604 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-serving-cert\") pod \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " Jul 07 02:20:39.637655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.637619 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-service-ca\") pod \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " Jul 07 02:20:39.638020 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.637671 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-config\") pod \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\" (UID: \"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb\") " Jul 07 02:20:39.638093 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.638062 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" (UID: "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:20:39.638093 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.638075 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-service-ca" (OuterVolumeSpecName: "service-ca") pod "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" (UID: "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:20:39.638317 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.638092 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-config" (OuterVolumeSpecName: "console-config") pod "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" (UID: "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:20:39.638406 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.638387 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" (UID: "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:20:39.639748 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.639721 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" (UID: "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 02:20:39.639860 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.639754 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" (UID: "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 02:20:39.639860 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.639791 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-kube-api-access-q8tq7" (OuterVolumeSpecName: "kube-api-access-q8tq7") pod "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" (UID: "5bc4427c-e383-4e5e-8dfb-42d28d24b6bb"). InnerVolumeSpecName "kube-api-access-q8tq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:20:39.739380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.739294 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8tq7\" (UniqueName: \"kubernetes.io/projected/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-kube-api-access-q8tq7\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:39.739380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.739326 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-oauth-config\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:39.739380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.739335 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-oauth-serving-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:39.739380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.739343 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-trusted-ca-bundle\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:39.739380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.739351 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-serving-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:39.739380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.739361 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-service-ca\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:39.739380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:39.739369 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb-console-config\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:20:40.467443 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:40.467414 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9fcbf6949-x7q8x" Jul 07 02:20:40.467443 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:40.467424 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9fcbf6949-x7q8x" event={"ID":"5bc4427c-e383-4e5e-8dfb-42d28d24b6bb","Type":"ContainerDied","Data":"7f3221cc6a33eeb07c6f2e8c7ed2a7d0f1d2d592985b2a90097ac33cd5bced87"} Jul 07 02:20:40.467957 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:40.467483 2559 scope.go:117] "RemoveContainer" containerID="7cae48c63d920208b66c12479ceae8f68dd9d67e942b3a5a42d0f032e80c1df0" Jul 07 02:20:40.469285 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:40.469247 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" event={"ID":"419c705b-aaa6-4472-a85a-57d2ce84f40f","Type":"ContainerStarted","Data":"753782e47a11d07c2bdb986524a5bbfc3bbf48ed6d7ef21180338d414e000a9e"} Jul 07 02:20:40.488305 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:40.488256 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-z4nd2" podStartSLOduration=2.448725094 podStartE2EDuration="28.488239126s" totalImagesPullingTime="26.039514032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:40.486211897 +0000 UTC m=+657.236633548" watchObservedRunningTime="2026-07-07 02:20:40.488239126 +0000 UTC m=+657.238660778" Jul 07 02:20:40.502442 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:40.502396 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9fcbf6949-x7q8x"] Jul 07 02:20:40.508524 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:40.508497 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9fcbf6949-x7q8x"] Jul 07 02:20:41.818274 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:41.818239 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" path="/var/lib/kubelet/pods/5bc4427c-e383-4e5e-8dfb-42d28d24b6bb/volumes" Jul 07 02:20:52.527830 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.527789 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp"] Jul 07 02:20:52.528259 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.528233 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" containerName="manager" Jul 07 02:20:52.528259 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.528251 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" containerName="manager" Jul 07 02:20:52.528374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.528276 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" containerName="console" Jul 07 02:20:52.528374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.528282 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" containerName="console" Jul 07 02:20:52.528374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.528343 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bc4427c-e383-4e5e-8dfb-42d28d24b6bb" containerName="console" Jul 07 02:20:52.528374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.528354 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b690989d-9d8f-44a1-acfe-a4f528760dfa" containerName="manager" Jul 07 02:20:52.544430 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.544404 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.544906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.544886 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp"] Jul 07 02:20:52.546818 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.546799 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bq54s\"" Jul 07 02:20:52.649733 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649702 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.649733 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649737 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq5mr\" (UniqueName: \"kubernetes.io/projected/f2e7d479-6c21-4872-b695-ac491c96d1e6-kube-api-access-tq5mr\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.649965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649758 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.649965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649779 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.649965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649795 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.649965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.649965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2e7d479-6c21-4872-b695-ac491c96d1e6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.649965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649945 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.650189 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.649994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751467 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq5mr\" (UniqueName: \"kubernetes.io/projected/f2e7d479-6c21-4872-b695-ac491c96d1e6-kube-api-access-tq5mr\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751516 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751543 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751610 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.751651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751643 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2e7d479-6c21-4872-b695-ac491c96d1e6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.752005 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751695 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.752005 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.751991 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.752097 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.752056 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.752198 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.752128 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.752277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.752216 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.752404 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.752388 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2e7d479-6c21-4872-b695-ac491c96d1e6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.753720 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.753692 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.754096 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.754074 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.760561 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.760531 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2e7d479-6c21-4872-b695-ac491c96d1e6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.760674 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.760580 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq5mr\" (UniqueName: \"kubernetes.io/projected/f2e7d479-6c21-4872-b695-ac491c96d1e6-kube-api-access-tq5mr\") pod \"maas-default-gateway-openshift-default-56654dcf7-hvnvp\" (UID: \"f2e7d479-6c21-4872-b695-ac491c96d1e6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.854653 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.854590 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:52.985776 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.985753 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp"] Jul 07 02:20:52.988054 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:20:52.988028 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e7d479_6c21_4872_b695_ac491c96d1e6.slice/crio-e9b55cf1e22496e266e699268f7d71f6f17da7df5928d70ee94946fb90a8e317 WatchSource:0}: Error finding container e9b55cf1e22496e266e699268f7d71f6f17da7df5928d70ee94946fb90a8e317: Status 404 returned error can't find the container with id e9b55cf1e22496e266e699268f7d71f6f17da7df5928d70ee94946fb90a8e317 Jul 07 02:20:52.990643 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.990612 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:20:52.990695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.990677 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:20:52.990729 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:52.990707 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:20:53.539249 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:53.539214 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" event={"ID":"f2e7d479-6c21-4872-b695-ac491c96d1e6","Type":"ContainerStarted","Data":"38a05e25fe9f5365a5284ffda684d49daf34b9d6d7cf3023600cb03e7e594800"} Jul 07 02:20:53.539630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:53.539262 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" event={"ID":"f2e7d479-6c21-4872-b695-ac491c96d1e6","Type":"ContainerStarted","Data":"e9b55cf1e22496e266e699268f7d71f6f17da7df5928d70ee94946fb90a8e317"} Jul 07 02:20:53.564247 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:53.564205 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" podStartSLOduration=1.564189614 podStartE2EDuration="1.564189614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:20:53.56264563 +0000 UTC m=+670.313067283" watchObservedRunningTime="2026-07-07 02:20:53.564189614 +0000 UTC m=+670.314611266" Jul 07 02:20:53.854789 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:53.854704 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:54.860216 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:54.860187 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:55.549077 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:55.549045 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:20:55.549877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:20:55.549858 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:21:07.920477 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:07.920442 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvzzw"] Jul 07 02:21:07.942959 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:07.942919 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvzzw"] Jul 07 02:21:07.943108 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:07.943059 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:07.945562 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:07.945534 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 02:21:08.018744 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.018714 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvzzw"] Jul 07 02:21:08.089810 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.089775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-config-file\") pod \"limitador-limitador-64c94fd64f-vvzzw\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:08.089810 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.089807 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgfs5\" (UniqueName: \"kubernetes.io/projected/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-kube-api-access-dgfs5\") pod \"limitador-limitador-64c94fd64f-vvzzw\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:08.191148 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.191061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-config-file\") pod \"limitador-limitador-64c94fd64f-vvzzw\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:08.191148 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.191096 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgfs5\" (UniqueName: \"kubernetes.io/projected/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-kube-api-access-dgfs5\") pod \"limitador-limitador-64c94fd64f-vvzzw\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:08.191794 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.191769 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-config-file\") pod \"limitador-limitador-64c94fd64f-vvzzw\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:08.199479 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.199450 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgfs5\" (UniqueName: \"kubernetes.io/projected/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-kube-api-access-dgfs5\") pod \"limitador-limitador-64c94fd64f-vvzzw\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:08.255286 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.255252 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:08.465329 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.465305 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvzzw"] Jul 07 02:21:08.467464 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:21:08.467437 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fe3684_a4a5_4a8a_9f7d_c72208dc0f8b.slice/crio-9e8bdc5854e2caab800aa11321a4e29481569a281d05344b0ed581e51891ee1e WatchSource:0}: Error finding container 9e8bdc5854e2caab800aa11321a4e29481569a281d05344b0ed581e51891ee1e: Status 404 returned error can't find the container with id 9e8bdc5854e2caab800aa11321a4e29481569a281d05344b0ed581e51891ee1e Jul 07 02:21:08.615625 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.615587 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" event={"ID":"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b","Type":"ContainerStarted","Data":"9e8bdc5854e2caab800aa11321a4e29481569a281d05344b0ed581e51891ee1e"} Jul 07 02:21:08.758022 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.757946 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-287zp"] Jul 07 02:21:08.835637 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.835604 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-287zp"] Jul 07 02:21:08.835802 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.835708 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-287zp" Jul 07 02:21:08.838203 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.838179 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7cf92\"" Jul 07 02:21:08.875673 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.875651 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wj4jf"] Jul 07 02:21:08.888117 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.888095 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wj4jf"] Jul 07 02:21:08.888258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.888198 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wj4jf" Jul 07 02:21:08.999475 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.999442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr95k\" (UniqueName: \"kubernetes.io/projected/b0bda471-e4de-46f4-852b-a7082d123139-kube-api-access-mr95k\") pod \"authorino-58cd955675-287zp\" (UID: \"b0bda471-e4de-46f4-852b-a7082d123139\") " pod="kuadrant-system/authorino-58cd955675-287zp" Jul 07 02:21:08.999939 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:08.999489 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr27g\" (UniqueName: \"kubernetes.io/projected/75e5db46-3c7c-4e4b-9192-21f84b966a1a-kube-api-access-xr27g\") pod \"authorino-75d556bd77-wj4jf\" (UID: \"75e5db46-3c7c-4e4b-9192-21f84b966a1a\") " pod="kuadrant-system/authorino-75d556bd77-wj4jf" Jul 07 02:21:09.100780 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.100698 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr95k\" (UniqueName: \"kubernetes.io/projected/b0bda471-e4de-46f4-852b-a7082d123139-kube-api-access-mr95k\") pod \"authorino-58cd955675-287zp\" (UID: \"b0bda471-e4de-46f4-852b-a7082d123139\") " pod="kuadrant-system/authorino-58cd955675-287zp" Jul 07 02:21:09.100780 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.100751 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr27g\" (UniqueName: \"kubernetes.io/projected/75e5db46-3c7c-4e4b-9192-21f84b966a1a-kube-api-access-xr27g\") pod \"authorino-75d556bd77-wj4jf\" (UID: \"75e5db46-3c7c-4e4b-9192-21f84b966a1a\") " pod="kuadrant-system/authorino-75d556bd77-wj4jf" Jul 07 02:21:09.108373 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.108345 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr95k\" (UniqueName: \"kubernetes.io/projected/b0bda471-e4de-46f4-852b-a7082d123139-kube-api-access-mr95k\") pod \"authorino-58cd955675-287zp\" (UID: \"b0bda471-e4de-46f4-852b-a7082d123139\") " pod="kuadrant-system/authorino-58cd955675-287zp" Jul 07 02:21:09.109208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.109187 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr27g\" (UniqueName: \"kubernetes.io/projected/75e5db46-3c7c-4e4b-9192-21f84b966a1a-kube-api-access-xr27g\") pod \"authorino-75d556bd77-wj4jf\" (UID: \"75e5db46-3c7c-4e4b-9192-21f84b966a1a\") " pod="kuadrant-system/authorino-75d556bd77-wj4jf" Jul 07 02:21:09.146973 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.146936 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-287zp" Jul 07 02:21:09.197615 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.197585 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wj4jf" Jul 07 02:21:09.310666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.310538 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-287zp"] Jul 07 02:21:09.322728 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:21:09.322695 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bda471_e4de_46f4_852b_a7082d123139.slice/crio-4aee944022e5d07698f8e03ac4e574a5c8fbfaa13e20370863daf0a8aec65978 WatchSource:0}: Error finding container 4aee944022e5d07698f8e03ac4e574a5c8fbfaa13e20370863daf0a8aec65978: Status 404 returned error can't find the container with id 4aee944022e5d07698f8e03ac4e574a5c8fbfaa13e20370863daf0a8aec65978 Jul 07 02:21:09.408508 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.408477 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wj4jf"] Jul 07 02:21:09.412898 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:21:09.412831 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e5db46_3c7c_4e4b_9192_21f84b966a1a.slice/crio-b9f067b3ff56970f7380ed6f68efede84efa4063854c8def80e121f5973c2a2d WatchSource:0}: Error finding container b9f067b3ff56970f7380ed6f68efede84efa4063854c8def80e121f5973c2a2d: Status 404 returned error can't find the container with id b9f067b3ff56970f7380ed6f68efede84efa4063854c8def80e121f5973c2a2d Jul 07 02:21:09.625381 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.625334 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wj4jf" event={"ID":"75e5db46-3c7c-4e4b-9192-21f84b966a1a","Type":"ContainerStarted","Data":"b9f067b3ff56970f7380ed6f68efede84efa4063854c8def80e121f5973c2a2d"} Jul 07 02:21:09.627174 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:09.627124 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-287zp" event={"ID":"b0bda471-e4de-46f4-852b-a7082d123139","Type":"ContainerStarted","Data":"4aee944022e5d07698f8e03ac4e574a5c8fbfaa13e20370863daf0a8aec65978"} Jul 07 02:21:14.660916 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.660874 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-287zp" event={"ID":"b0bda471-e4de-46f4-852b-a7082d123139","Type":"ContainerStarted","Data":"4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf"} Jul 07 02:21:14.662347 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.662315 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wj4jf" event={"ID":"75e5db46-3c7c-4e4b-9192-21f84b966a1a","Type":"ContainerStarted","Data":"6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683"} Jul 07 02:21:14.666961 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.666937 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" event={"ID":"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b","Type":"ContainerStarted","Data":"ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c"} Jul 07 02:21:14.667108 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.667088 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:14.679208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.679148 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-287zp" podStartSLOduration=2.1483193 podStartE2EDuration="6.679132678s" totalImagesPullingTime="4.530813378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:14.677814583 +0000 UTC m=+691.428236237" watchObservedRunningTime="2026-07-07 02:21:14.679132678 +0000 UTC m=+691.429554330" Jul 07 02:21:14.695472 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.695422 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" podStartSLOduration=2.300972471 podStartE2EDuration="7.695403417s" totalImagesPullingTime="5.394430946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:14.692479661 +0000 UTC m=+691.442901343" watchObservedRunningTime="2026-07-07 02:21:14.695403417 +0000 UTC m=+691.445825070" Jul 07 02:21:14.708373 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.708324 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wj4jf" podStartSLOduration=2.297240636 podStartE2EDuration="6.708308453s" totalImagesPullingTime="4.411067817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:14.70705172 +0000 UTC m=+691.457473373" watchObservedRunningTime="2026-07-07 02:21:14.708308453 +0000 UTC m=+691.458730105" Jul 07 02:21:14.733121 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:14.733086 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-287zp"] Jul 07 02:21:16.677403 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:16.677361 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-287zp" podUID="b0bda471-e4de-46f4-852b-a7082d123139" containerName="authorino" containerID="cri-o://4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf" gracePeriod=30 Jul 07 02:21:16.926252 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:16.926217 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-287zp" Jul 07 02:21:16.977221 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:16.977114 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr95k\" (UniqueName: \"kubernetes.io/projected/b0bda471-e4de-46f4-852b-a7082d123139-kube-api-access-mr95k\") pod \"b0bda471-e4de-46f4-852b-a7082d123139\" (UID: \"b0bda471-e4de-46f4-852b-a7082d123139\") " Jul 07 02:21:16.979262 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:16.979237 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0bda471-e4de-46f4-852b-a7082d123139-kube-api-access-mr95k" (OuterVolumeSpecName: "kube-api-access-mr95k") pod "b0bda471-e4de-46f4-852b-a7082d123139" (UID: "b0bda471-e4de-46f4-852b-a7082d123139"). InnerVolumeSpecName "kube-api-access-mr95k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:21:17.077646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.077612 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr95k\" (UniqueName: \"kubernetes.io/projected/b0bda471-e4de-46f4-852b-a7082d123139-kube-api-access-mr95k\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:21:17.683807 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.683769 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0bda471-e4de-46f4-852b-a7082d123139" containerID="4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf" exitCode=0 Jul 07 02:21:17.683807 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.683810 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-287zp" Jul 07 02:21:17.684308 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.683840 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-287zp" event={"ID":"b0bda471-e4de-46f4-852b-a7082d123139","Type":"ContainerDied","Data":"4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf"} Jul 07 02:21:17.684308 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.683879 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-287zp" event={"ID":"b0bda471-e4de-46f4-852b-a7082d123139","Type":"ContainerDied","Data":"4aee944022e5d07698f8e03ac4e574a5c8fbfaa13e20370863daf0a8aec65978"} Jul 07 02:21:17.684308 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.683898 2559 scope.go:117] "RemoveContainer" containerID="4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf" Jul 07 02:21:17.692541 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.692524 2559 scope.go:117] "RemoveContainer" containerID="4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf" Jul 07 02:21:17.692804 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:21:17.692786 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf\": container with ID starting with 4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf not found: ID does not exist" containerID="4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf" Jul 07 02:21:17.692855 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.692814 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf"} err="failed to get container status \"4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf\": rpc error: code = NotFound desc = could not find container \"4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf\": container with ID starting with 4f646d694324aba8db3b8050136fa35341fe16b5805504bdfae3374d87a294bf not found: ID does not exist" Jul 07 02:21:17.704542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.704515 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-287zp"] Jul 07 02:21:17.713031 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.713004 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-287zp"] Jul 07 02:21:17.818861 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:17.818827 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0bda471-e4de-46f4-852b-a7082d123139" path="/var/lib/kubelet/pods/b0bda471-e4de-46f4-852b-a7082d123139/volumes" Jul 07 02:21:23.404784 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:23.404749 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvzzw"] Jul 07 02:21:23.405444 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:23.405016 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" podUID="46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" containerName="limitador" containerID="cri-o://ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c" gracePeriod=30 Jul 07 02:21:23.405822 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:23.405763 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:23.735404 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:21:23.735375 2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fe3684_a4a5_4a8a_9f7d_c72208dc0f8b.slice/crio-ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c.scope\": RecentStats: unable to find data in memory cache]" Jul 07 02:21:23.956583 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:23.956559 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:24.035986 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.035901 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-config-file\") pod \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " Jul 07 02:21:24.035986 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.035940 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgfs5\" (UniqueName: \"kubernetes.io/projected/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-kube-api-access-dgfs5\") pod \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\" (UID: \"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b\") " Jul 07 02:21:24.036381 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.036356 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-config-file" (OuterVolumeSpecName: "config-file") pod "46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" (UID: "46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:21:24.038044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.038023 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-kube-api-access-dgfs5" (OuterVolumeSpecName: "kube-api-access-dgfs5") pod "46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" (UID: "46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b"). InnerVolumeSpecName "kube-api-access-dgfs5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:21:24.139371 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.137092 2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-config-file\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:21:24.139371 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.137124 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgfs5\" (UniqueName: \"kubernetes.io/projected/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b-kube-api-access-dgfs5\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:21:24.724744 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.724703 2559 generic.go:356] "Generic (PLEG): container finished" podID="46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" containerID="ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c" exitCode=0 Jul 07 02:21:24.725139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.724766 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" Jul 07 02:21:24.725139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.724784 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" event={"ID":"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b","Type":"ContainerDied","Data":"ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c"} Jul 07 02:21:24.725139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.724867 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vvzzw" event={"ID":"46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b","Type":"ContainerDied","Data":"9e8bdc5854e2caab800aa11321a4e29481569a281d05344b0ed581e51891ee1e"} Jul 07 02:21:24.725139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.724891 2559 scope.go:117] "RemoveContainer" containerID="ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c" Jul 07 02:21:24.733795 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.733778 2559 scope.go:117] "RemoveContainer" containerID="ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c" Jul 07 02:21:24.734068 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:21:24.734050 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c\": container with ID starting with ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c not found: ID does not exist" containerID="ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c" Jul 07 02:21:24.734113 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.734077 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c"} err="failed to get container status \"ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c\": rpc error: code = NotFound desc = could not find container \"ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c\": container with ID starting with ac162db03c337c5f64c64d7b8add3f051ef85cc48a223ca7bf0b505af24c6b2c not found: ID does not exist" Jul 07 02:21:24.746266 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.746238 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvzzw"] Jul 07 02:21:24.749661 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:24.749638 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vvzzw"] Jul 07 02:21:25.817896 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:25.817863 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" path="/var/lib/kubelet/pods/46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b/volumes" Jul 07 02:21:26.399483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.399449 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-vznhk"] Jul 07 02:21:26.399980 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.399966 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0bda471-e4de-46f4-852b-a7082d123139" containerName="authorino" Jul 07 02:21:26.400024 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.399984 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0bda471-e4de-46f4-852b-a7082d123139" containerName="authorino" Jul 07 02:21:26.400024 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.400014 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" containerName="limitador" Jul 07 02:21:26.400086 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.400023 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" containerName="limitador" Jul 07 02:21:26.400135 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.400124 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="46fe3684-a4a5-4a8a-9f7d-c72208dc0f8b" containerName="limitador" Jul 07 02:21:26.400212 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.400144 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0bda471-e4de-46f4-852b-a7082d123139" containerName="authorino" Jul 07 02:21:26.404659 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.404640 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.407135 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.407115 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-d8nc7\"" Jul 07 02:21:26.407279 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.407262 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 02:21:26.411230 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.411206 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-vznhk"] Jul 07 02:21:26.457496 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.457467 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmkst\" (UniqueName: \"kubernetes.io/projected/eead3123-e443-4af4-afbb-e0f4d6455ecb-kube-api-access-qmkst\") pod \"postgres-69fb4594cf-vznhk\" (UID: \"eead3123-e443-4af4-afbb-e0f4d6455ecb\") " pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.457689 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.457542 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eead3123-e443-4af4-afbb-e0f4d6455ecb-data\") pod \"postgres-69fb4594cf-vznhk\" (UID: \"eead3123-e443-4af4-afbb-e0f4d6455ecb\") " pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.559039 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.558992 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eead3123-e443-4af4-afbb-e0f4d6455ecb-data\") pod \"postgres-69fb4594cf-vznhk\" (UID: \"eead3123-e443-4af4-afbb-e0f4d6455ecb\") " pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.559255 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.559093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmkst\" (UniqueName: \"kubernetes.io/projected/eead3123-e443-4af4-afbb-e0f4d6455ecb-kube-api-access-qmkst\") pod \"postgres-69fb4594cf-vznhk\" (UID: \"eead3123-e443-4af4-afbb-e0f4d6455ecb\") " pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.559404 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.559383 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eead3123-e443-4af4-afbb-e0f4d6455ecb-data\") pod \"postgres-69fb4594cf-vznhk\" (UID: \"eead3123-e443-4af4-afbb-e0f4d6455ecb\") " pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.567878 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.567852 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmkst\" (UniqueName: \"kubernetes.io/projected/eead3123-e443-4af4-afbb-e0f4d6455ecb-kube-api-access-qmkst\") pod \"postgres-69fb4594cf-vznhk\" (UID: \"eead3123-e443-4af4-afbb-e0f4d6455ecb\") " pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.716503 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.716410 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:26.852550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:26.852520 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-vznhk"] Jul 07 02:21:26.856468 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:21:26.856442 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeead3123_e443_4af4_afbb_e0f4d6455ecb.slice/crio-61f3732d1ce0ec76ea75c57407dca4ef169efbc2665fd599da6c15aa039c38a2 WatchSource:0}: Error finding container 61f3732d1ce0ec76ea75c57407dca4ef169efbc2665fd599da6c15aa039c38a2: Status 404 returned error can't find the container with id 61f3732d1ce0ec76ea75c57407dca4ef169efbc2665fd599da6c15aa039c38a2 Jul 07 02:21:27.746207 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:27.746104 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-vznhk" event={"ID":"eead3123-e443-4af4-afbb-e0f4d6455ecb","Type":"ContainerStarted","Data":"61f3732d1ce0ec76ea75c57407dca4ef169efbc2665fd599da6c15aa039c38a2"} Jul 07 02:21:32.778287 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:32.778247 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-vznhk" event={"ID":"eead3123-e443-4af4-afbb-e0f4d6455ecb","Type":"ContainerStarted","Data":"b7ec2388e8826977c9581ed3b6b54ff1232ffec800a0e12cb039065aa1788eed"} Jul 07 02:21:32.778765 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:32.778323 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:32.797225 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:32.797177 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-vznhk" podStartSLOduration=1.876373859 podStartE2EDuration="6.797139947s" totalImagesPullingTime="4.920766088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:32.796325252 +0000 UTC m=+709.546746909" watchObservedRunningTime="2026-07-07 02:21:32.797139947 +0000 UTC m=+709.547561599" Jul 07 02:21:38.812956 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:38.812886 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-vznhk" Jul 07 02:21:39.360787 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.360742 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5dlz"] Jul 07 02:21:39.363371 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.363351 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5dlz" Jul 07 02:21:39.371932 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.371908 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5dlz"] Jul 07 02:21:39.480016 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.479977 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qztmh\" (UniqueName: \"kubernetes.io/projected/0334ca82-1b7e-495f-b838-65623d5867ea-kube-api-access-qztmh\") pod \"authorino-5d4b67c446-n5dlz\" (UID: \"0334ca82-1b7e-495f-b838-65623d5867ea\") " pod="kuadrant-system/authorino-5d4b67c446-n5dlz" Jul 07 02:21:39.548465 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.548434 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5dlz"] Jul 07 02:21:39.548708 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:21:39.548690 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qztmh], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-n5dlz" podUID="0334ca82-1b7e-495f-b838-65623d5867ea" Jul 07 02:21:39.580674 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.580643 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qztmh\" (UniqueName: \"kubernetes.io/projected/0334ca82-1b7e-495f-b838-65623d5867ea-kube-api-access-qztmh\") pod \"authorino-5d4b67c446-n5dlz\" (UID: \"0334ca82-1b7e-495f-b838-65623d5867ea\") " pod="kuadrant-system/authorino-5d4b67c446-n5dlz" Jul 07 02:21:39.598362 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.598333 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qztmh\" (UniqueName: \"kubernetes.io/projected/0334ca82-1b7e-495f-b838-65623d5867ea-kube-api-access-qztmh\") pod \"authorino-5d4b67c446-n5dlz\" (UID: \"0334ca82-1b7e-495f-b838-65623d5867ea\") " pod="kuadrant-system/authorino-5d4b67c446-n5dlz" Jul 07 02:21:39.814518 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.814489 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5dlz" Jul 07 02:21:39.821101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.821076 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5dlz" Jul 07 02:21:39.882926 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.882887 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qztmh\" (UniqueName: \"kubernetes.io/projected/0334ca82-1b7e-495f-b838-65623d5867ea-kube-api-access-qztmh\") pod \"0334ca82-1b7e-495f-b838-65623d5867ea\" (UID: \"0334ca82-1b7e-495f-b838-65623d5867ea\") " Jul 07 02:21:39.885053 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.885034 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0334ca82-1b7e-495f-b838-65623d5867ea-kube-api-access-qztmh" (OuterVolumeSpecName: "kube-api-access-qztmh") pod "0334ca82-1b7e-495f-b838-65623d5867ea" (UID: "0334ca82-1b7e-495f-b838-65623d5867ea"). InnerVolumeSpecName "kube-api-access-qztmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:21:39.984004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:39.983963 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qztmh\" (UniqueName: \"kubernetes.io/projected/0334ca82-1b7e-495f-b838-65623d5867ea-kube-api-access-qztmh\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:21:40.819206 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:40.819177 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-n5dlz" Jul 07 02:21:40.849493 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:40.849461 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5dlz"] Jul 07 02:21:40.851749 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:40.851724 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-n5dlz"] Jul 07 02:21:41.818312 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:41.818279 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0334ca82-1b7e-495f-b838-65623d5867ea" path="/var/lib/kubelet/pods/0334ca82-1b7e-495f-b838-65623d5867ea/volumes" Jul 07 02:21:43.557521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.557482 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wj4jf"] Jul 07 02:21:43.557952 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.557710 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wj4jf" podUID="75e5db46-3c7c-4e4b-9192-21f84b966a1a" containerName="authorino" containerID="cri-o://6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683" gracePeriod=30 Jul 07 02:21:43.833537 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.833474 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wj4jf" Jul 07 02:21:43.835106 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.835079 2559 generic.go:356] "Generic (PLEG): container finished" podID="75e5db46-3c7c-4e4b-9192-21f84b966a1a" containerID="6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683" exitCode=0 Jul 07 02:21:43.835237 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.835114 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wj4jf" Jul 07 02:21:43.835237 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.835120 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wj4jf" event={"ID":"75e5db46-3c7c-4e4b-9192-21f84b966a1a","Type":"ContainerDied","Data":"6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683"} Jul 07 02:21:43.835237 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.835144 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wj4jf" event={"ID":"75e5db46-3c7c-4e4b-9192-21f84b966a1a","Type":"ContainerDied","Data":"b9f067b3ff56970f7380ed6f68efede84efa4063854c8def80e121f5973c2a2d"} Jul 07 02:21:43.835237 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.835190 2559 scope.go:117] "RemoveContainer" containerID="6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683" Jul 07 02:21:43.843509 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.843492 2559 scope.go:117] "RemoveContainer" containerID="6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683" Jul 07 02:21:43.843775 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:21:43.843753 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683\": container with ID starting with 6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683 not found: ID does not exist" containerID="6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683" Jul 07 02:21:43.843870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.843782 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683"} err="failed to get container status \"6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683\": rpc error: code = NotFound desc = could not find container \"6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683\": container with ID starting with 6de10896950c42f431673aff1dc8b9eb73b37836b12dcdad9d3c1acb00759683 not found: ID does not exist" Jul 07 02:21:43.918593 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.918562 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr27g\" (UniqueName: \"kubernetes.io/projected/75e5db46-3c7c-4e4b-9192-21f84b966a1a-kube-api-access-xr27g\") pod \"75e5db46-3c7c-4e4b-9192-21f84b966a1a\" (UID: \"75e5db46-3c7c-4e4b-9192-21f84b966a1a\") " Jul 07 02:21:43.920557 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:43.920531 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e5db46-3c7c-4e4b-9192-21f84b966a1a-kube-api-access-xr27g" (OuterVolumeSpecName: "kube-api-access-xr27g") pod "75e5db46-3c7c-4e4b-9192-21f84b966a1a" (UID: "75e5db46-3c7c-4e4b-9192-21f84b966a1a"). InnerVolumeSpecName "kube-api-access-xr27g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:21:44.019833 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:44.019796 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr27g\" (UniqueName: \"kubernetes.io/projected/75e5db46-3c7c-4e4b-9192-21f84b966a1a-kube-api-access-xr27g\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:21:44.163769 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:44.163735 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wj4jf"] Jul 07 02:21:44.172330 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:44.172299 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wj4jf"] Jul 07 02:21:45.819094 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:45.819062 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e5db46-3c7c-4e4b-9192-21f84b966a1a" path="/var/lib/kubelet/pods/75e5db46-3c7c-4e4b-9192-21f84b966a1a/volumes" Jul 07 02:21:47.891356 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.891321 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-dd47dcc8-fmgvp"] Jul 07 02:21:47.891733 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.891670 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e5db46-3c7c-4e4b-9192-21f84b966a1a" containerName="authorino" Jul 07 02:21:47.891733 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.891682 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e5db46-3c7c-4e4b-9192-21f84b966a1a" containerName="authorino" Jul 07 02:21:47.891807 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.891745 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e5db46-3c7c-4e4b-9192-21f84b966a1a" containerName="authorino" Jul 07 02:21:47.905482 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.905452 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:47.908222 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.908193 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 02:21:47.908375 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.908197 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cl2fm\"" Jul 07 02:21:47.909039 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.909019 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 02:21:47.909188 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:47.909036 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-dd47dcc8-fmgvp"] Jul 07 02:21:48.058696 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.058660 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85vc9\" (UniqueName: \"kubernetes.io/projected/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-kube-api-access-85vc9\") pod \"maas-controller-dd47dcc8-fmgvp\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:48.058855 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.058747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-webhook-cert\") pod \"maas-controller-dd47dcc8-fmgvp\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:48.159706 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.159673 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-webhook-cert\") pod \"maas-controller-dd47dcc8-fmgvp\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:48.159875 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.159721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85vc9\" (UniqueName: \"kubernetes.io/projected/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-kube-api-access-85vc9\") pod \"maas-controller-dd47dcc8-fmgvp\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:48.162105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.162078 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-webhook-cert\") pod \"maas-controller-dd47dcc8-fmgvp\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:48.167208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.167184 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85vc9\" (UniqueName: \"kubernetes.io/projected/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-kube-api-access-85vc9\") pod \"maas-controller-dd47dcc8-fmgvp\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:48.218006 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.217976 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:48.351056 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.351030 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-dd47dcc8-fmgvp"] Jul 07 02:21:48.353045 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:21:48.353021 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0fac4f6_6c6f_4da6_97cf_17ee4f396468.slice/crio-21044231d0e15b820bad10bf0e44478e54e6d2dbc5543b241bb972e59a76906e WatchSource:0}: Error finding container 21044231d0e15b820bad10bf0e44478e54e6d2dbc5543b241bb972e59a76906e: Status 404 returned error can't find the container with id 21044231d0e15b820bad10bf0e44478e54e6d2dbc5543b241bb972e59a76906e Jul 07 02:21:48.862037 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:48.861997 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" event={"ID":"b0fac4f6-6c6f-4da6-97cf-17ee4f396468","Type":"ContainerStarted","Data":"21044231d0e15b820bad10bf0e44478e54e6d2dbc5543b241bb972e59a76906e"} Jul 07 02:21:50.876317 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:50.876277 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" event={"ID":"b0fac4f6-6c6f-4da6-97cf-17ee4f396468","Type":"ContainerStarted","Data":"2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498"} Jul 07 02:21:50.876708 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:50.876374 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:21:50.896313 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:21:50.896266 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" podStartSLOduration=1.9424449209999999 podStartE2EDuration="3.896250572s" totalImagesPullingTime="1.953805651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:21:50.8939975 +0000 UTC m=+727.644419156" watchObservedRunningTime="2026-07-07 02:21:50.896250572 +0000 UTC m=+727.646672224" Jul 07 02:22:01.887135 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:01.887102 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:22:02.449912 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.449876 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-dd47dcc8-fmgvp"] Jul 07 02:22:02.450093 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.450059 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" podUID="b0fac4f6-6c6f-4da6-97cf-17ee4f396468" containerName="manager" containerID="cri-o://2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498" gracePeriod=30 Jul 07 02:22:02.691410 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.691385 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:22:02.784297 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.784219 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85vc9\" (UniqueName: \"kubernetes.io/projected/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-kube-api-access-85vc9\") pod \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " Jul 07 02:22:02.784297 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.784251 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-webhook-cert\") pod \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\" (UID: \"b0fac4f6-6c6f-4da6-97cf-17ee4f396468\") " Jul 07 02:22:02.786420 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.786388 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b0fac4f6-6c6f-4da6-97cf-17ee4f396468" (UID: "b0fac4f6-6c6f-4da6-97cf-17ee4f396468"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 02:22:02.786522 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.786470 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-kube-api-access-85vc9" (OuterVolumeSpecName: "kube-api-access-85vc9") pod "b0fac4f6-6c6f-4da6-97cf-17ee4f396468" (UID: "b0fac4f6-6c6f-4da6-97cf-17ee4f396468"). InnerVolumeSpecName "kube-api-access-85vc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:22:02.885434 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.885403 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85vc9\" (UniqueName: \"kubernetes.io/projected/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-kube-api-access-85vc9\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:22:02.885434 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.885430 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0fac4f6-6c6f-4da6-97cf-17ee4f396468-webhook-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:22:02.936966 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.936932 2559 generic.go:356] "Generic (PLEG): container finished" podID="b0fac4f6-6c6f-4da6-97cf-17ee4f396468" containerID="2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498" exitCode=0 Jul 07 02:22:02.937374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.936980 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" Jul 07 02:22:02.937374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.937019 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" event={"ID":"b0fac4f6-6c6f-4da6-97cf-17ee4f396468","Type":"ContainerDied","Data":"2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498"} Jul 07 02:22:02.937374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.937075 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-dd47dcc8-fmgvp" event={"ID":"b0fac4f6-6c6f-4da6-97cf-17ee4f396468","Type":"ContainerDied","Data":"21044231d0e15b820bad10bf0e44478e54e6d2dbc5543b241bb972e59a76906e"} Jul 07 02:22:02.937374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.937098 2559 scope.go:117] "RemoveContainer" containerID="2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498" Jul 07 02:22:02.946530 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.946513 2559 scope.go:117] "RemoveContainer" containerID="2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498" Jul 07 02:22:02.946794 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:22:02.946770 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498\": container with ID starting with 2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498 not found: ID does not exist" containerID="2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498" Jul 07 02:22:02.946897 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.946799 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498"} err="failed to get container status \"2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498\": rpc error: code = NotFound desc = could not find container \"2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498\": container with ID starting with 2aeffecb7dbfcc8c273c518bf22be4a74c220fd5f759c0a62be18e231b9de498 not found: ID does not exist" Jul 07 02:22:02.962202 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.960138 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-dd47dcc8-fmgvp"] Jul 07 02:22:02.967483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:02.967444 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-dd47dcc8-fmgvp"] Jul 07 02:22:03.819725 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:03.819695 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0fac4f6-6c6f-4da6-97cf-17ee4f396468" path="/var/lib/kubelet/pods/b0fac4f6-6c6f-4da6-97cf-17ee4f396468/volumes" Jul 07 02:22:21.616442 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.616410 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-cc8d47df9-lmg2v"] Jul 07 02:22:21.616906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.616772 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fac4f6-6c6f-4da6-97cf-17ee4f396468" containerName="manager" Jul 07 02:22:21.616906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.616783 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fac4f6-6c6f-4da6-97cf-17ee4f396468" containerName="manager" Jul 07 02:22:21.616906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.616844 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0fac4f6-6c6f-4da6-97cf-17ee4f396468" containerName="manager" Jul 07 02:22:21.619013 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.618997 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.623102 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.623077 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 02:22:21.623254 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.623170 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2kkqn\"" Jul 07 02:22:21.662026 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.661992 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-cc8d47df9-lmg2v"] Jul 07 02:22:21.752259 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.752223 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4309c267-cdfe-4259-8b42-86c8779dabde-maas-api-tls\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.752447 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.752277 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4309c267-cdfe-4259-8b42-86c8779dabde-openshift-service-ca\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.752447 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.752350 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b2nt\" (UniqueName: \"kubernetes.io/projected/4309c267-cdfe-4259-8b42-86c8779dabde-kube-api-access-5b2nt\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.853759 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.853729 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b2nt\" (UniqueName: \"kubernetes.io/projected/4309c267-cdfe-4259-8b42-86c8779dabde-kube-api-access-5b2nt\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.853933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.853810 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4309c267-cdfe-4259-8b42-86c8779dabde-maas-api-tls\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.853933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.853841 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4309c267-cdfe-4259-8b42-86c8779dabde-openshift-service-ca\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.854732 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.854702 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4309c267-cdfe-4259-8b42-86c8779dabde-openshift-service-ca\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.856345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.856328 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4309c267-cdfe-4259-8b42-86c8779dabde-maas-api-tls\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.873731 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.873663 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b2nt\" (UniqueName: \"kubernetes.io/projected/4309c267-cdfe-4259-8b42-86c8779dabde-kube-api-access-5b2nt\") pod \"maas-api-cc8d47df9-lmg2v\" (UID: \"4309c267-cdfe-4259-8b42-86c8779dabde\") " pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:21.930176 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:21.930125 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:22.075750 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.075726 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-cc8d47df9-lmg2v"] Jul 07 02:22:22.078259 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:22:22.078219 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4309c267_cdfe_4259_8b42_86c8779dabde.slice/crio-a5f3036d31299a218b51bd586189fde9d3cb868e513e971796427a250e5b2590 WatchSource:0}: Error finding container a5f3036d31299a218b51bd586189fde9d3cb868e513e971796427a250e5b2590: Status 404 returned error can't find the container with id a5f3036d31299a218b51bd586189fde9d3cb868e513e971796427a250e5b2590 Jul 07 02:22:22.550124 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.550090 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zslkb"] Jul 07 02:22:22.553390 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.553368 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.574287 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.574263 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 02:22:22.583009 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.582978 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zslkb"] Jul 07 02:22:22.628905 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.628866 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-p52nl"] Jul 07 02:22:22.632144 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.632119 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:22.636012 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.635972 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dlvth\"" Jul 07 02:22:22.650874 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.650842 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-p52nl"] Jul 07 02:22:22.660839 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.660811 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c358ad22-3d77-4895-b7af-f7e230131dac-plugins-config-volume\") pod \"payload-processing-58f86c7798-zslkb\" (UID: \"c358ad22-3d77-4895-b7af-f7e230131dac\") " pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.661007 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.660852 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn9q2\" (UniqueName: \"kubernetes.io/projected/c358ad22-3d77-4895-b7af-f7e230131dac-kube-api-access-dn9q2\") pod \"payload-processing-58f86c7798-zslkb\" (UID: \"c358ad22-3d77-4895-b7af-f7e230131dac\") " pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.762297 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.762262 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79t5g\" (UniqueName: \"kubernetes.io/projected/e1caa62c-7e05-4161-a0e8-a6e619f7c16f-kube-api-access-79t5g\") pod \"payload-pre-processing-7964457696-p52nl\" (UID: \"e1caa62c-7e05-4161-a0e8-a6e619f7c16f\") " pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:22.762474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.762328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c358ad22-3d77-4895-b7af-f7e230131dac-plugins-config-volume\") pod \"payload-processing-58f86c7798-zslkb\" (UID: \"c358ad22-3d77-4895-b7af-f7e230131dac\") " pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.762474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.762361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn9q2\" (UniqueName: \"kubernetes.io/projected/c358ad22-3d77-4895-b7af-f7e230131dac-kube-api-access-dn9q2\") pod \"payload-processing-58f86c7798-zslkb\" (UID: \"c358ad22-3d77-4895-b7af-f7e230131dac\") " pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.762474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.762380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1caa62c-7e05-4161-a0e8-a6e619f7c16f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-p52nl\" (UID: \"e1caa62c-7e05-4161-a0e8-a6e619f7c16f\") " pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:22.763110 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.763089 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c358ad22-3d77-4895-b7af-f7e230131dac-plugins-config-volume\") pod \"payload-processing-58f86c7798-zslkb\" (UID: \"c358ad22-3d77-4895-b7af-f7e230131dac\") " pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.774320 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.774293 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn9q2\" (UniqueName: \"kubernetes.io/projected/c358ad22-3d77-4895-b7af-f7e230131dac-kube-api-access-dn9q2\") pod \"payload-processing-58f86c7798-zslkb\" (UID: \"c358ad22-3d77-4895-b7af-f7e230131dac\") " pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.863318 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.863232 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1caa62c-7e05-4161-a0e8-a6e619f7c16f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-p52nl\" (UID: \"e1caa62c-7e05-4161-a0e8-a6e619f7c16f\") " pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:22.863479 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.863381 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79t5g\" (UniqueName: \"kubernetes.io/projected/e1caa62c-7e05-4161-a0e8-a6e619f7c16f-kube-api-access-79t5g\") pod \"payload-pre-processing-7964457696-p52nl\" (UID: \"e1caa62c-7e05-4161-a0e8-a6e619f7c16f\") " pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:22.863564 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.863529 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:22.864335 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.864310 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1caa62c-7e05-4161-a0e8-a6e619f7c16f-plugins-config-volume\") pod \"payload-pre-processing-7964457696-p52nl\" (UID: \"e1caa62c-7e05-4161-a0e8-a6e619f7c16f\") " pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:22.878692 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.878661 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79t5g\" (UniqueName: \"kubernetes.io/projected/e1caa62c-7e05-4161-a0e8-a6e619f7c16f-kube-api-access-79t5g\") pod \"payload-pre-processing-7964457696-p52nl\" (UID: \"e1caa62c-7e05-4161-a0e8-a6e619f7c16f\") " pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:22.948545 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:22.948510 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:23.030555 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:23.030472 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zslkb"] Jul 07 02:22:23.048094 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:23.048072 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-zslkb" event={"ID":"c358ad22-3d77-4895-b7af-f7e230131dac","Type":"ContainerStarted","Data":"12e5f1d44f0ebd9394f8342c3268d10a9c36c7cf5378401fb2ba84be1ae278c9"} Jul 07 02:22:23.049451 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:23.049428 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-cc8d47df9-lmg2v" event={"ID":"4309c267-cdfe-4259-8b42-86c8779dabde","Type":"ContainerStarted","Data":"a5f3036d31299a218b51bd586189fde9d3cb868e513e971796427a250e5b2590"} Jul 07 02:22:23.124006 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:23.123913 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-p52nl"] Jul 07 02:22:23.128025 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:22:23.127962 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1caa62c_7e05_4161_a0e8_a6e619f7c16f.slice/crio-2bdb47d72c7956b7c4b1d8927dd524a6eac611323861d3afa343e67158c10f0f WatchSource:0}: Error finding container 2bdb47d72c7956b7c4b1d8927dd524a6eac611323861d3afa343e67158c10f0f: Status 404 returned error can't find the container with id 2bdb47d72c7956b7c4b1d8927dd524a6eac611323861d3afa343e67158c10f0f Jul 07 02:22:24.074732 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:24.074651 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" event={"ID":"e1caa62c-7e05-4161-a0e8-a6e619f7c16f","Type":"ContainerStarted","Data":"2bdb47d72c7956b7c4b1d8927dd524a6eac611323861d3afa343e67158c10f0f"} Jul 07 02:22:25.081248 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.081142 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-zslkb" event={"ID":"c358ad22-3d77-4895-b7af-f7e230131dac","Type":"ContainerStarted","Data":"151d1713b76c33667c4509d9e5cb4ced9a1be9f11b30f79d96f7c4720fc88329"} Jul 07 02:22:25.081714 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.081292 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:25.083171 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.083128 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-cc8d47df9-lmg2v" event={"ID":"4309c267-cdfe-4259-8b42-86c8779dabde","Type":"ContainerStarted","Data":"165b5d9d78bf05f3a8f8a5412e66a7055c740a6945ad5e56c6a075b0e763e54c"} Jul 07 02:22:25.083295 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.083210 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:25.084716 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.084674 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" event={"ID":"e1caa62c-7e05-4161-a0e8-a6e619f7c16f","Type":"ContainerStarted","Data":"b650d1372d9c66c0c2fdf445be17a875850758c271a5233f8b1cacf8d89b8ae1"} Jul 07 02:22:25.084848 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.084806 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:25.131743 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.131695 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-cc8d47df9-lmg2v" podStartSLOduration=1.2592888 podStartE2EDuration="4.131680007s" totalImagesPullingTime="2.872391207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:25.129232064 +0000 UTC m=+761.879653717" watchObservedRunningTime="2026-07-07 02:22:25.131680007 +0000 UTC m=+761.882101659" Jul 07 02:22:25.131903 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:25.131782 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-zslkb" podStartSLOduration=1.209778675 podStartE2EDuration="3.131776429s" totalImagesPullingTime="1.921997754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:25.10493136 +0000 UTC m=+761.855353012" watchObservedRunningTime="2026-07-07 02:22:25.131776429 +0000 UTC m=+761.882198084" Jul 07 02:22:31.094780 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.094753 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-cc8d47df9-lmg2v" Jul 07 02:22:31.117175 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.117132 2559 generic.go:356] "Generic (PLEG): container finished" podID="f2e7d479-6c21-4872-b695-ac491c96d1e6" containerID="38a05e25fe9f5365a5284ffda684d49daf34b9d6d7cf3023600cb03e7e594800" exitCode=0 Jul 07 02:22:31.117324 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.117206 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" event={"ID":"f2e7d479-6c21-4872-b695-ac491c96d1e6","Type":"ContainerDied","Data":"38a05e25fe9f5365a5284ffda684d49daf34b9d6d7cf3023600cb03e7e594800"} Jul 07 02:22:31.117573 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.117560 2559 scope.go:117] "RemoveContainer" containerID="38a05e25fe9f5365a5284ffda684d49daf34b9d6d7cf3023600cb03e7e594800" Jul 07 02:22:31.118627 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.118591 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:22:31.118692 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.118673 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:22:31.118727 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.118718 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:22:31.121780 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:31.121744 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" podStartSLOduration=7.2993028859999995 podStartE2EDuration="9.12173123s" totalImagesPullingTime="1.822428344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:22:25.161351406 +0000 UTC m=+761.911773059" watchObservedRunningTime="2026-07-07 02:22:31.12173123 +0000 UTC m=+767.872152883" Jul 07 02:22:32.124704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:32.124667 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" event={"ID":"f2e7d479-6c21-4872-b695-ac491c96d1e6","Type":"ContainerStarted","Data":"06f40c158c1f9f45e018052ee01a7522934a3e946fc32f712785f3c6e97d5267"} Jul 07 02:22:32.855618 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:32.855583 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:22:32.855967 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:32.855942 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hvnvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 02:22:32.856169 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:32.856117 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" podUID="f2e7d479-6c21-4872-b695-ac491c96d1e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 02:22:33.855231 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:33.855195 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hvnvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 02:22:33.855729 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:33.855270 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" podUID="f2e7d479-6c21-4872-b695-ac491c96d1e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 02:22:34.855958 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:34.855922 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hvnvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 02:22:34.856562 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:34.856532 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" podUID="f2e7d479-6c21-4872-b695-ac491c96d1e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 02:22:35.855358 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:35.855322 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hvnvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 07 02:22:35.855572 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:35.855392 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" podUID="f2e7d479-6c21-4872-b695-ac491c96d1e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 07 02:22:36.092026 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:36.091994 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-zslkb" Jul 07 02:22:36.092489 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:36.092187 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-p52nl" Jul 07 02:22:36.860018 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:36.859135 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:22:36.861191 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:36.860988 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:22:36.865667 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:22:36.865646 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hvnvp" Jul 07 02:23:10.327055 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.327027 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz"] Jul 07 02:23:10.330489 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.330471 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.335560 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.335540 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bzvlw\"" Jul 07 02:23:10.336134 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.336111 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 02:23:10.336569 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.336547 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 02:23:10.336569 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.336564 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 02:23:10.342111 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.342089 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz"] Jul 07 02:23:10.402391 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.402359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c66f979a-97c8-4a34-b3c6-576605cb1a08-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.402736 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.402705 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.402864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.402766 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.402864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.402838 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.402864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.402876 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9pf2\" (UniqueName: \"kubernetes.io/projected/c66f979a-97c8-4a34-b3c6-576605cb1a08-kube-api-access-z9pf2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.403307 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.402921 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504179 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c66f979a-97c8-4a34-b3c6-576605cb1a08-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504245 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504285 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504319 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9pf2\" (UniqueName: \"kubernetes.io/projected/c66f979a-97c8-4a34-b3c6-576605cb1a08-kube-api-access-z9pf2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504374 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504674 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504649 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504796 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504710 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.504796 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.504756 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.506506 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.506476 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c66f979a-97c8-4a34-b3c6-576605cb1a08-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.506643 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.506623 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c66f979a-97c8-4a34-b3c6-576605cb1a08-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.514070 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.514050 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9pf2\" (UniqueName: \"kubernetes.io/projected/c66f979a-97c8-4a34-b3c6-576605cb1a08-kube-api-access-z9pf2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5mbz\" (UID: \"c66f979a-97c8-4a34-b3c6-576605cb1a08\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.642474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.642393 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:10.786023 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:10.785927 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz"] Jul 07 02:23:10.788716 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:23:10.788677 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc66f979a_97c8_4a34_b3c6_576605cb1a08.slice/crio-9d509d760548bdf6f03c3f1934de286fef4a74274fd998f759859d13c4a6cfbf WatchSource:0}: Error finding container 9d509d760548bdf6f03c3f1934de286fef4a74274fd998f759859d13c4a6cfbf: Status 404 returned error can't find the container with id 9d509d760548bdf6f03c3f1934de286fef4a74274fd998f759859d13c4a6cfbf Jul 07 02:23:11.339225 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:11.339189 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" event={"ID":"c66f979a-97c8-4a34-b3c6-576605cb1a08","Type":"ContainerStarted","Data":"9d509d760548bdf6f03c3f1934de286fef4a74274fd998f759859d13c4a6cfbf"} Jul 07 02:23:12.346232 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:12.346194 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" event={"ID":"c66f979a-97c8-4a34-b3c6-576605cb1a08","Type":"ContainerStarted","Data":"c80810e8ac8941a54b8380c65cfdbcdf9774d5dc35de63e9b5274de4a6aa1a4d"} Jul 07 02:23:12.474820 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:12.474766 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" podStartSLOduration=1.473895095 podStartE2EDuration="2.474744083s" totalImagesPullingTime="1.000848988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:23:12.47279684 +0000 UTC m=+809.223218492" watchObservedRunningTime="2026-07-07 02:23:12.474744083 +0000 UTC m=+809.225165763" Jul 07 02:23:20.642903 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:20.642865 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:20.643316 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:20.642962 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:20.655939 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:20.655913 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:21.404925 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:21.404892 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5mbz" Jul 07 02:23:35.196327 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.196292 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-px787"] Jul 07 02:23:35.200902 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.200884 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.213175 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.213147 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 02:23:35.230384 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.230360 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.230466 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.230392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97743ae2-f73a-4494-8ccf-356f9281b0b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.230466 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.230433 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64lbg\" (UniqueName: \"kubernetes.io/projected/97743ae2-f73a-4494-8ccf-356f9281b0b5-kube-api-access-64lbg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.230544 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.230468 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.230544 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.230485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.230544 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.230508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.331496 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.331447 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64lbg\" (UniqueName: \"kubernetes.io/projected/97743ae2-f73a-4494-8ccf-356f9281b0b5-kube-api-access-64lbg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.331678 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.331511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.331678 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.331535 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.331678 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.331557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.331815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.331704 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.331815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.331754 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97743ae2-f73a-4494-8ccf-356f9281b0b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.331973 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.331954 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.332039 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.332019 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.332087 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.332066 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.333871 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.333854 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97743ae2-f73a-4494-8ccf-356f9281b0b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.334053 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.334037 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97743ae2-f73a-4494-8ccf-356f9281b0b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.358123 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.358099 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64lbg\" (UniqueName: \"kubernetes.io/projected/97743ae2-f73a-4494-8ccf-356f9281b0b5-kube-api-access-64lbg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-px787\" (UID: \"97743ae2-f73a-4494-8ccf-356f9281b0b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.359275 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.359252 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-px787"] Jul 07 02:23:35.511127 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.511041 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:35.659125 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:35.659091 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-px787"] Jul 07 02:23:35.660661 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:23:35.660633 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97743ae2_f73a_4494_8ccf_356f9281b0b5.slice/crio-db1c7af00e36f4acc9f453123e671558e368258df8af149c3d904e58f1a96359 WatchSource:0}: Error finding container db1c7af00e36f4acc9f453123e671558e368258df8af149c3d904e58f1a96359: Status 404 returned error can't find the container with id db1c7af00e36f4acc9f453123e671558e368258df8af149c3d904e58f1a96359 Jul 07 02:23:36.475407 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:36.475372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" event={"ID":"97743ae2-f73a-4494-8ccf-356f9281b0b5","Type":"ContainerStarted","Data":"59c38c2d83e9d7fa5a0011cc7ee217752d1c9f6377ef80b3470c903c761edef5"} Jul 07 02:23:36.475407 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:36.475423 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" event={"ID":"97743ae2-f73a-4494-8ccf-356f9281b0b5","Type":"ContainerStarted","Data":"db1c7af00e36f4acc9f453123e671558e368258df8af149c3d904e58f1a96359"} Jul 07 02:23:36.501173 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:36.501102 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" podStartSLOduration=1.307654652 podStartE2EDuration="1.501087515s" totalImagesPullingTime="193.432863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:23:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:23:36.499653803 +0000 UTC m=+833.250075458" watchObservedRunningTime="2026-07-07 02:23:36.501087515 +0000 UTC m=+833.251509167" Jul 07 02:23:45.511764 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:45.511716 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:45.511764 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:45.511785 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:45.524379 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:45.524354 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:23:45.535613 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:23:45.535587 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-px787" Jul 07 02:24:29.526697 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.526664 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dbbb754f5-7lxs7"] Jul 07 02:24:29.530925 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.530894 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.535324 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.535296 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7cf92\"" Jul 07 02:24:29.536135 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.536110 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 02:24:29.543330 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.543307 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dbbb754f5-7lxs7"] Jul 07 02:24:29.617775 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.617743 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6836ceaa-e27d-4b28-badf-77714eaf9c39-tls-cert\") pod \"authorino-6dbbb754f5-7lxs7\" (UID: \"6836ceaa-e27d-4b28-badf-77714eaf9c39\") " pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.617956 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.617815 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jth5\" (UniqueName: \"kubernetes.io/projected/6836ceaa-e27d-4b28-badf-77714eaf9c39-kube-api-access-5jth5\") pod \"authorino-6dbbb754f5-7lxs7\" (UID: \"6836ceaa-e27d-4b28-badf-77714eaf9c39\") " pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.719086 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.719028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6836ceaa-e27d-4b28-badf-77714eaf9c39-tls-cert\") pod \"authorino-6dbbb754f5-7lxs7\" (UID: \"6836ceaa-e27d-4b28-badf-77714eaf9c39\") " pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.719318 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.719149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jth5\" (UniqueName: \"kubernetes.io/projected/6836ceaa-e27d-4b28-badf-77714eaf9c39-kube-api-access-5jth5\") pod \"authorino-6dbbb754f5-7lxs7\" (UID: \"6836ceaa-e27d-4b28-badf-77714eaf9c39\") " pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.721492 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.721461 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6836ceaa-e27d-4b28-badf-77714eaf9c39-tls-cert\") pod \"authorino-6dbbb754f5-7lxs7\" (UID: \"6836ceaa-e27d-4b28-badf-77714eaf9c39\") " pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.732463 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.732438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jth5\" (UniqueName: \"kubernetes.io/projected/6836ceaa-e27d-4b28-badf-77714eaf9c39-kube-api-access-5jth5\") pod \"authorino-6dbbb754f5-7lxs7\" (UID: \"6836ceaa-e27d-4b28-badf-77714eaf9c39\") " pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.842661 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.842576 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" Jul 07 02:24:29.988620 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:29.988594 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dbbb754f5-7lxs7"] Jul 07 02:24:29.989673 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:24:29.989649 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6836ceaa_e27d_4b28_badf_77714eaf9c39.slice/crio-8019fe20bb39863e5c9a6df826fcefd16e2fbdfa05f42eb37a8ffa4bb275a86f WatchSource:0}: Error finding container 8019fe20bb39863e5c9a6df826fcefd16e2fbdfa05f42eb37a8ffa4bb275a86f: Status 404 returned error can't find the container with id 8019fe20bb39863e5c9a6df826fcefd16e2fbdfa05f42eb37a8ffa4bb275a86f Jul 07 02:24:30.760864 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:30.760830 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" event={"ID":"6836ceaa-e27d-4b28-badf-77714eaf9c39","Type":"ContainerStarted","Data":"dbb628f2a4d64875da5ad82bcd4df4e047bac19d05c5b8631139188cd9ca048e"} Jul 07 02:24:30.761257 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:30.760874 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" event={"ID":"6836ceaa-e27d-4b28-badf-77714eaf9c39","Type":"ContainerStarted","Data":"8019fe20bb39863e5c9a6df826fcefd16e2fbdfa05f42eb37a8ffa4bb275a86f"} Jul 07 02:24:30.789987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:24:30.789889 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dbbb754f5-7lxs7" podStartSLOduration=1.263759858 podStartE2EDuration="1.78987266s" totalImagesPullingTime="526.112802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:24:30.787346799 +0000 UTC m=+887.537768453" watchObservedRunningTime="2026-07-07 02:24:30.78987266 +0000 UTC m=+887.540294313" Jul 07 02:37:39.609705 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.607671 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb"] Jul 07 02:37:39.611240 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.611223 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.613856 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.613837 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d01c49f5-openshift-default-dockercfg-phxlp\"" Jul 07 02:37:39.633585 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.633562 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb"] Jul 07 02:37:39.710806 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.710769 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-envoy\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.710806 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.710803 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-token\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.711008 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.710862 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-socket\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.711008 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.710964 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-data\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.711118 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.711014 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-credential-socket\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.711118 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.711048 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-podinfo\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.711262 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.711122 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-certs\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.711262 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.711148 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk4m4\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-kube-api-access-xk4m4\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.711262 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.711197 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istiod-ca-cert\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811708 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811674 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-envoy\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811708 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811710 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-token\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811732 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-socket\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811784 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-data\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811827 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-credential-socket\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811851 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-podinfo\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811894 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-certs\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811916 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk4m4\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-kube-api-access-xk4m4\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.811945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.811943 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istiod-ca-cert\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.812354 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.812329 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-data\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.812413 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.812362 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-credential-socket\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.812413 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.812383 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-socket\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.812524 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.812463 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-certs\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.812969 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.812945 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istiod-ca-cert\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.814103 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.814079 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-envoy\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.814346 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.814327 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-podinfo\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.820805 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.820775 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-token\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.821131 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.821112 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk4m4\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-kube-api-access-xk4m4\") pod \"e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:39.921168 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:39.921125 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:40.062972 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:40.062947 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb"] Jul 07 02:37:40.064557 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:37:40.064531 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9746335_4693_41a9_8d6d_b72c7b0fbcda.slice/crio-06c4c1b872ea2a8c64596cb1a720d2b5e63b4c8b96baaf26ceff47a240fb42a2 WatchSource:0}: Error finding container 06c4c1b872ea2a8c64596cb1a720d2b5e63b4c8b96baaf26ceff47a240fb42a2: Status 404 returned error can't find the container with id 06c4c1b872ea2a8c64596cb1a720d2b5e63b4c8b96baaf26ceff47a240fb42a2 Jul 07 02:37:40.067192 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:40.067139 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:37:40.067300 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:40.067220 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:37:40.067300 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:40.067250 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:37:40.081654 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:40.081632 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" event={"ID":"b9746335-4693-41a9-8d6d-b72c7b0fbcda","Type":"ContainerStarted","Data":"06c4c1b872ea2a8c64596cb1a720d2b5e63b4c8b96baaf26ceff47a240fb42a2"} Jul 07 02:37:41.088445 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:41.088408 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" event={"ID":"b9746335-4693-41a9-8d6d-b72c7b0fbcda","Type":"ContainerStarted","Data":"644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f"} Jul 07 02:37:41.114093 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:41.114041 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" podStartSLOduration=2.114025795 podStartE2EDuration="2.114025795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:37:41.111214468 +0000 UTC m=+1677.861636120" watchObservedRunningTime="2026-07-07 02:37:41.114025795 +0000 UTC m=+1677.864447448" Jul 07 02:37:41.921889 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:41.921855 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:41.926637 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:41.926613 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:42.093947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:42.093916 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:42.094978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:42.094959 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:46.124976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:46.124944 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb"] Jul 07 02:37:46.125495 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:46.125115 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" podUID="b9746335-4693-41a9-8d6d-b72c7b0fbcda" containerName="istio-proxy" containerID="cri-o://644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f" gracePeriod=30 Jul 07 02:37:52.301565 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.301524 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86"] Jul 07 02:37:52.306821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.306799 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.310724 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.310694 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-144e32df-openshift-default-dockercfg-c9mlx\"" Jul 07 02:37:52.327149 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.327118 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86"] Jul 07 02:37:52.433263 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433215 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-certs\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433276 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-data\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433296 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-envoy\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433322 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-podinfo\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-token\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrmqt\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-kube-api-access-vrmqt\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433438 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433435 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/face9c1c-a36f-4ec3-9592-8297952c3f8f-istiod-ca-cert\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433475 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-credential-socket\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.433646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.433497 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-socket\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.534861 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.534820 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-data\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.534864 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-envoy\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.534898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-podinfo\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.534915 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-token\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.534940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrmqt\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-kube-api-access-vrmqt\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.534964 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/face9c1c-a36f-4ec3-9592-8297952c3f8f-istiod-ca-cert\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535219 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-credential-socket\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535264 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-data\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535271 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-socket\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535397 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-certs\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535545 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-socket\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535631 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-credential-socket\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535667 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/face9c1c-a36f-4ec3-9592-8297952c3f8f-istiod-ca-cert\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.536559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.535691 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-certs\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.537955 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.537922 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-envoy\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.538120 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.538099 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-podinfo\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.542962 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.542939 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-token\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.543376 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.543344 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrmqt\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-kube-api-access-vrmqt\") pod \"e2e-ait-144e32df-openshift-default-94dfcb849-kcp86\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.618248 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.618223 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:52.782611 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.782585 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86"] Jul 07 02:37:52.784540 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:37:52.784509 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podface9c1c_a36f_4ec3_9592_8297952c3f8f.slice/crio-f249b2002fd1db166ed445747947634e9fa6e74ef8036449229aa151b9b79635 WatchSource:0}: Error finding container f249b2002fd1db166ed445747947634e9fa6e74ef8036449229aa151b9b79635: Status 404 returned error can't find the container with id f249b2002fd1db166ed445747947634e9fa6e74ef8036449229aa151b9b79635 Jul 07 02:37:52.786734 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.786697 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:37:52.786867 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.786782 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:37:52.786867 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.786827 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:37:52.788400 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.788380 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:52.939826 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.939793 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk4m4\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-kube-api-access-xk4m4\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.939985 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.939851 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-socket\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.939985 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.939890 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-credential-socket\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.939985 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.939914 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-certs\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.939985 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.939950 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-token\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.940225 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940018 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-podinfo\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.940225 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940062 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istiod-ca-cert\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.940225 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940095 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-envoy\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.940225 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940129 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-data\") pod \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\" (UID: \"b9746335-4693-41a9-8d6d-b72c7b0fbcda\") " Jul 07 02:37:52.940419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940220 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:37:52.940419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940234 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:37:52.940419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940258 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:37:52.940582 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940519 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:37:52.940642 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940592 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:52.940642 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940612 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:52.940642 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.940628 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:52.941067 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.941022 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-data" (OuterVolumeSpecName: "istio-data") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:37:52.942571 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.942546 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:37:52.942767 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.942733 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:37:52.942870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.942779 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-token" (OuterVolumeSpecName: "istio-token") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:37:52.942870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:52.942795 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-kube-api-access-xk4m4" (OuterVolumeSpecName: "kube-api-access-xk4m4") pod "b9746335-4693-41a9-8d6d-b72c7b0fbcda" (UID: "b9746335-4693-41a9-8d6d-b72c7b0fbcda"). InnerVolumeSpecName "kube-api-access-xk4m4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:37:53.041911 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.041836 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk4m4\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-kube-api-access-xk4m4\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:53.041911 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.041867 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:53.041911 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.041881 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:53.041911 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.041893 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:53.041911 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.041909 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:53.042191 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.041922 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9746335-4693-41a9-8d6d-b72c7b0fbcda-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:37:53.156516 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.156483 2559 generic.go:356] "Generic (PLEG): container finished" podID="b9746335-4693-41a9-8d6d-b72c7b0fbcda" containerID="644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f" exitCode=0 Jul 07 02:37:53.156516 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.156527 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" Jul 07 02:37:53.156738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.156567 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" event={"ID":"b9746335-4693-41a9-8d6d-b72c7b0fbcda","Type":"ContainerDied","Data":"644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f"} Jul 07 02:37:53.156738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.156632 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb" event={"ID":"b9746335-4693-41a9-8d6d-b72c7b0fbcda","Type":"ContainerDied","Data":"06c4c1b872ea2a8c64596cb1a720d2b5e63b4c8b96baaf26ceff47a240fb42a2"} Jul 07 02:37:53.156738 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.156666 2559 scope.go:117] "RemoveContainer" containerID="644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f" Jul 07 02:37:53.162222 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.162191 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" event={"ID":"face9c1c-a36f-4ec3-9592-8297952c3f8f","Type":"ContainerStarted","Data":"3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba"} Jul 07 02:37:53.162349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.162287 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" event={"ID":"face9c1c-a36f-4ec3-9592-8297952c3f8f","Type":"ContainerStarted","Data":"f249b2002fd1db166ed445747947634e9fa6e74ef8036449229aa151b9b79635"} Jul 07 02:37:53.172940 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.172917 2559 scope.go:117] "RemoveContainer" containerID="644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f" Jul 07 02:37:53.173297 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:37:53.173277 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f\": container with ID starting with 644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f not found: ID does not exist" containerID="644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f" Jul 07 02:37:53.173369 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.173305 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f"} err="failed to get container status \"644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f\": rpc error: code = NotFound desc = could not find container \"644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f\": container with ID starting with 644ecdc093f1cc73dcda2ad724e1d4233b76e28d0be45d98941d56bed917c70f not found: ID does not exist" Jul 07 02:37:53.187412 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.187369 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" podStartSLOduration=1.18735385 podStartE2EDuration="1.18735385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:37:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:37:53.184317186 +0000 UTC m=+1689.934738864" watchObservedRunningTime="2026-07-07 02:37:53.18735385 +0000 UTC m=+1689.937775506" Jul 07 02:37:53.202644 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.202615 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb"] Jul 07 02:37:53.218577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.218556 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d01c49f5-openshift-default-7bc4bff9ff-qv4cb"] Jul 07 02:37:53.619420 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.619385 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:53.624170 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.624137 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:53.818528 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:53.818453 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9746335-4693-41a9-8d6d-b72c7b0fbcda" path="/var/lib/kubelet/pods/b9746335-4693-41a9-8d6d-b72c7b0fbcda/volumes" Jul 07 02:37:54.169297 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:54.169265 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:37:54.170132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:37:54.170115 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:38:04.223189 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:04.223136 2559 generic.go:356] "Generic (PLEG): container finished" podID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerID="3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba" exitCode=0 Jul 07 02:38:04.223680 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:04.223215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" event={"ID":"face9c1c-a36f-4ec3-9592-8297952c3f8f","Type":"ContainerDied","Data":"3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba"} Jul 07 02:38:04.223766 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:04.223748 2559 scope.go:117] "RemoveContainer" containerID="3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba" Jul 07 02:38:04.224802 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:04.224772 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:38:04.224899 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:04.224837 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:38:04.224899 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:04.224869 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:38:04.557581 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:04.557450 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86"] Jul 07 02:38:05.230816 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:05.230779 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" event={"ID":"face9c1c-a36f-4ec3-9592-8297952c3f8f","Type":"ContainerStarted","Data":"b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51"} Jul 07 02:38:05.231346 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:05.230817 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerName="istio-proxy" containerID="cri-o://b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51" gracePeriod=30 Jul 07 02:38:09.169914 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:09.169883 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:38:10.508560 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.508532 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:38:10.591752 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591710 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-certs\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591752 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591762 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-envoy\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591784 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-podinfo\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591800 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-socket\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591825 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-credential-socket\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591876 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-data\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591907 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-token\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591935 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrmqt\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-kube-api-access-vrmqt\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.591987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.591968 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/face9c1c-a36f-4ec3-9592-8297952c3f8f-istiod-ca-cert\") pod \"face9c1c-a36f-4ec3-9592-8297952c3f8f\" (UID: \"face9c1c-a36f-4ec3-9592-8297952c3f8f\") " Jul 07 02:38:10.594416 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.594386 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:10.594607 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.594578 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:10.594708 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.594649 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/face9c1c-a36f-4ec3-9592-8297952c3f8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:38:10.594777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.594723 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:10.595082 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.595053 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:10.596324 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.596298 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:10.596441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.596422 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:38:10.596723 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.596696 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:38:10.596723 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.596707 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-kube-api-access-vrmqt" (OuterVolumeSpecName: "kube-api-access-vrmqt") pod "face9c1c-a36f-4ec3-9592-8297952c3f8f" (UID: "face9c1c-a36f-4ec3-9592-8297952c3f8f"). InnerVolumeSpecName "kube-api-access-vrmqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:38:10.692690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692652 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrmqt\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-kube-api-access-vrmqt\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692683 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/face9c1c-a36f-4ec3-9592-8297952c3f8f-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692690 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692694 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692931 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692702 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692931 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692710 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692931 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692718 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692931 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692726 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692931 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692734 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.692931 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.692741 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/face9c1c-a36f-4ec3-9592-8297952c3f8f-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:10.714964 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.714931 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf"] Jul 07 02:38:10.715464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715442 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerName="istio-proxy" Jul 07 02:38:10.715464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715463 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerName="istio-proxy" Jul 07 02:38:10.715630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715472 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9746335-4693-41a9-8d6d-b72c7b0fbcda" containerName="istio-proxy" Jul 07 02:38:10.715630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715478 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9746335-4693-41a9-8d6d-b72c7b0fbcda" containerName="istio-proxy" Jul 07 02:38:10.715630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715494 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerName="istio-proxy" Jul 07 02:38:10.715630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715499 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerName="istio-proxy" Jul 07 02:38:10.715630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715561 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9746335-4693-41a9-8d6d-b72c7b0fbcda" containerName="istio-proxy" Jul 07 02:38:10.715630 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.715571 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerName="istio-proxy" Jul 07 02:38:10.718453 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.718435 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.720946 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.720926 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c2f066bd-openshift-default-dockercfg-gbx9f\"" Jul 07 02:38:10.732200 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.732175 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf"] Jul 07 02:38:10.793821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.793739 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-credential-socket\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.793821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.793777 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-socket\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.794002 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.793822 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-token\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.794002 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.793902 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-certs\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.794002 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.793926 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93170669-70d4-4c92-a897-5ad83cdc4cf2-istiod-ca-cert\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.794002 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.793962 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-podinfo\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.794002 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.793982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-envoy\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.794181 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.794026 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvjc2\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-kube-api-access-vvjc2\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.794181 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.794048 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-data\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894455 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894410 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-credential-socket\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894455 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-socket\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894495 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-token\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894559 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-certs\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894580 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93170669-70d4-4c92-a897-5ad83cdc4cf2-istiod-ca-cert\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-podinfo\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894713 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894671 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-envoy\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894715 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvjc2\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-kube-api-access-vvjc2\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-data\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894911 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-socket\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.894978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.894943 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-credential-socket\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.895372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.895348 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-certs\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.895464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.895388 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-data\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.895569 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.895550 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93170669-70d4-4c92-a897-5ad83cdc4cf2-istiod-ca-cert\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.897074 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.897053 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-podinfo\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.897121 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.897071 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-envoy\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.902779 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.902754 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-token\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:10.902954 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:10.902938 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvjc2\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-kube-api-access-vvjc2\") pod \"e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:11.031660 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.031630 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:11.167693 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.167667 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf"] Jul 07 02:38:11.169364 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:38:11.169340 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93170669_70d4_4c92_a897_5ad83cdc4cf2.slice/crio-71d2c8624df8fa21dc8781459189884d98feb8144eca4eabb5c675d9b5999a5e WatchSource:0}: Error finding container 71d2c8624df8fa21dc8781459189884d98feb8144eca4eabb5c675d9b5999a5e: Status 404 returned error can't find the container with id 71d2c8624df8fa21dc8781459189884d98feb8144eca4eabb5c675d9b5999a5e Jul 07 02:38:11.171546 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.171512 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:38:11.171625 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.171579 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:38:11.171625 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.171615 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:38:11.268602 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.268562 2559 generic.go:356] "Generic (PLEG): container finished" podID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerID="b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51" exitCode=0 Jul 07 02:38:11.268786 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.268610 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" Jul 07 02:38:11.268786 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.268623 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" event={"ID":"face9c1c-a36f-4ec3-9592-8297952c3f8f","Type":"ContainerDied","Data":"b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51"} Jul 07 02:38:11.268786 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.268687 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86" event={"ID":"face9c1c-a36f-4ec3-9592-8297952c3f8f","Type":"ContainerDied","Data":"f249b2002fd1db166ed445747947634e9fa6e74ef8036449229aa151b9b79635"} Jul 07 02:38:11.268786 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.268734 2559 scope.go:117] "RemoveContainer" containerID="b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51" Jul 07 02:38:11.270929 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.270843 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" event={"ID":"93170669-70d4-4c92-a897-5ad83cdc4cf2","Type":"ContainerStarted","Data":"7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141"} Jul 07 02:38:11.270929 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.270897 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" event={"ID":"93170669-70d4-4c92-a897-5ad83cdc4cf2","Type":"ContainerStarted","Data":"71d2c8624df8fa21dc8781459189884d98feb8144eca4eabb5c675d9b5999a5e"} Jul 07 02:38:11.278825 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.278808 2559 scope.go:117] "RemoveContainer" containerID="3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba" Jul 07 02:38:11.287280 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.287261 2559 scope.go:117] "RemoveContainer" containerID="b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51" Jul 07 02:38:11.287566 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:38:11.287517 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51\": container with ID starting with b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51 not found: ID does not exist" containerID="b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51" Jul 07 02:38:11.287640 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.287573 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51"} err="failed to get container status \"b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51\": rpc error: code = NotFound desc = could not find container \"b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51\": container with ID starting with b47f84d23eb67053e894fa77daa034f8bfb9a1cd8c50c8d17547ce8ac8616b51 not found: ID does not exist" Jul 07 02:38:11.287640 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.287592 2559 scope.go:117] "RemoveContainer" containerID="3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba" Jul 07 02:38:11.287842 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:38:11.287823 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba\": container with ID starting with 3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba not found: ID does not exist" containerID="3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba" Jul 07 02:38:11.287909 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.287847 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba"} err="failed to get container status \"3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba\": rpc error: code = NotFound desc = could not find container \"3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba\": container with ID starting with 3596ad8f500a4d80e3e0e3ab9f4e57c68c3d96b79b9560e427f2f56ef33e9bba not found: ID does not exist" Jul 07 02:38:11.295877 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.295825 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" podStartSLOduration=1.295807435 podStartE2EDuration="1.295807435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:38:11.29375911 +0000 UTC m=+1708.044180763" watchObservedRunningTime="2026-07-07 02:38:11.295807435 +0000 UTC m=+1708.046229089" Jul 07 02:38:11.309667 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.309638 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86"] Jul 07 02:38:11.314866 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.314844 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-144e32df-openshift-default-94dfcb849-kcp86"] Jul 07 02:38:11.820399 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:11.820364 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" path="/var/lib/kubelet/pods/face9c1c-a36f-4ec3-9592-8297952c3f8f/volumes" Jul 07 02:38:12.032296 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:12.032257 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:12.037728 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:12.037700 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:12.278111 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:12.278081 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:12.279134 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:12.279113 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:16.446548 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:16.446498 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf"] Jul 07 02:38:16.447064 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:16.446735 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" podUID="93170669-70d4-4c92-a897-5ad83cdc4cf2" containerName="istio-proxy" containerID="cri-o://7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141" gracePeriod=30 Jul 07 02:38:23.005050 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.005023 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:23.115968 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.115885 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvjc2\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-kube-api-access-vvjc2\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.115968 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.115926 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93170669-70d4-4c92-a897-5ad83cdc4cf2-istiod-ca-cert\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.115968 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.115968 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-socket\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.116258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116008 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-certs\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.116258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116038 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-credential-socket\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.116258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116057 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-token\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.116258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116099 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-envoy\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.116258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116128 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-data\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.116258 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116196 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-podinfo\") pod \"93170669-70d4-4c92-a897-5ad83cdc4cf2\" (UID: \"93170669-70d4-4c92-a897-5ad83cdc4cf2\") " Jul 07 02:38:23.116568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116334 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:23.116568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116356 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:23.116568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116376 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:23.116568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116421 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93170669-70d4-4c92-a897-5ad83cdc4cf2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:38:23.116777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116663 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.116777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116686 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.116777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116700 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.116777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.116715 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93170669-70d4-4c92-a897-5ad83cdc4cf2-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.117127 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.117105 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-data" (OuterVolumeSpecName: "istio-data") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:23.118408 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.118380 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:38:23.118528 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.118465 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-token" (OuterVolumeSpecName: "istio-token") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:38:23.118528 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.118493 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:38:23.118603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.118542 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-kube-api-access-vvjc2" (OuterVolumeSpecName: "kube-api-access-vvjc2") pod "93170669-70d4-4c92-a897-5ad83cdc4cf2" (UID: "93170669-70d4-4c92-a897-5ad83cdc4cf2"). InnerVolumeSpecName "kube-api-access-vvjc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:38:23.217953 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.217916 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvjc2\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-kube-api-access-vvjc2\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.217953 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.217947 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.217953 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.217958 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.218240 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.217966 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.218240 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.217976 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93170669-70d4-4c92-a897-5ad83cdc4cf2-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:38:23.352052 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.352013 2559 generic.go:356] "Generic (PLEG): container finished" podID="93170669-70d4-4c92-a897-5ad83cdc4cf2" containerID="7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141" exitCode=0 Jul 07 02:38:23.352294 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.352065 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" Jul 07 02:38:23.352294 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.352085 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" event={"ID":"93170669-70d4-4c92-a897-5ad83cdc4cf2","Type":"ContainerDied","Data":"7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141"} Jul 07 02:38:23.352294 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.352139 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf" event={"ID":"93170669-70d4-4c92-a897-5ad83cdc4cf2","Type":"ContainerDied","Data":"71d2c8624df8fa21dc8781459189884d98feb8144eca4eabb5c675d9b5999a5e"} Jul 07 02:38:23.352294 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.352188 2559 scope.go:117] "RemoveContainer" containerID="7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141" Jul 07 02:38:23.362252 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.362234 2559 scope.go:117] "RemoveContainer" containerID="7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141" Jul 07 02:38:23.362549 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:38:23.362527 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141\": container with ID starting with 7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141 not found: ID does not exist" containerID="7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141" Jul 07 02:38:23.362618 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.362559 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141"} err="failed to get container status \"7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141\": rpc error: code = NotFound desc = could not find container \"7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141\": container with ID starting with 7ad95cb81446c0c49d3f0fa96f75d3444a6ebe28de69ce3463b724f6fc4c1141 not found: ID does not exist" Jul 07 02:38:23.376452 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.376373 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf"] Jul 07 02:38:23.382600 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.382580 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c2f066bd-openshift-default-c567c64d7-vrgjf"] Jul 07 02:38:23.819543 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:38:23.819512 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93170669-70d4-4c92-a897-5ad83cdc4cf2" path="/var/lib/kubelet/pods/93170669-70d4-4c92-a897-5ad83cdc4cf2/volumes" Jul 07 02:39:57.840671 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.840642 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d"] Jul 07 02:39:57.841080 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.841026 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93170669-70d4-4c92-a897-5ad83cdc4cf2" containerName="istio-proxy" Jul 07 02:39:57.841080 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.841038 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="93170669-70d4-4c92-a897-5ad83cdc4cf2" containerName="istio-proxy" Jul 07 02:39:57.841177 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.841105 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="93170669-70d4-4c92-a897-5ad83cdc4cf2" containerName="istio-proxy" Jul 07 02:39:57.841177 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.841113 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="face9c1c-a36f-4ec3-9592-8297952c3f8f" containerName="istio-proxy" Jul 07 02:39:57.843336 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.843320 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.846120 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.846098 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3a3e4d5a-openshift-default-dockercfg-hh88f\"" Jul 07 02:39:57.869632 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.869601 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d"] Jul 07 02:39:57.902642 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.902609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-token\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.902821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.902656 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-credential-socket\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.902821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.902686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-certs\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.902943 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.902833 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-envoy\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.902943 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.902863 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istiod-ca-cert\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.903058 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.902996 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-podinfo\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.903058 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.903028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vqvc\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-kube-api-access-4vqvc\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.903058 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.903053 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-socket\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:57.903239 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:57.903085 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-data\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.003955 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.003913 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqvc\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-kube-api-access-4vqvc\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.003955 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.003960 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-socket\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.003980 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-data\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004009 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-token\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-credential-socket\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004053 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-certs\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004081 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-envoy\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istiod-ca-cert\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004224 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-podinfo\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004617 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004425 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-socket\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004617 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004605 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-credential-socket\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004727 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004631 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-data\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004780 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004723 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-certs\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.004922 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.004906 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istiod-ca-cert\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.006541 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.006507 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-envoy\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.006688 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.006667 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-podinfo\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.011950 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.011929 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-token\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.012213 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.012176 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vqvc\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-kube-api-access-4vqvc\") pod \"e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.153277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.153238 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:58.297359 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.297333 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d"] Jul 07 02:39:58.298752 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:39:58.298719 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fafd300_bb71_4e76_b8e1_ec00d9f3c696.slice/crio-f111cda67c6823aa9ea0cf92e86ae5d8e9e13dca2d71d24e5eddcbb79779430d WatchSource:0}: Error finding container f111cda67c6823aa9ea0cf92e86ae5d8e9e13dca2d71d24e5eddcbb79779430d: Status 404 returned error can't find the container with id f111cda67c6823aa9ea0cf92e86ae5d8e9e13dca2d71d24e5eddcbb79779430d Jul 07 02:39:58.300824 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.300793 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:39:58.300910 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.300857 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:39:58.300910 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.300889 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:39:58.865978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.865932 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" event={"ID":"2fafd300-bb71-4e76-b8e1-ec00d9f3c696","Type":"ContainerStarted","Data":"bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24"} Jul 07 02:39:58.866439 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.866013 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" event={"ID":"2fafd300-bb71-4e76-b8e1-ec00d9f3c696","Type":"ContainerStarted","Data":"f111cda67c6823aa9ea0cf92e86ae5d8e9e13dca2d71d24e5eddcbb79779430d"} Jul 07 02:39:58.925028 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:58.924973 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" podStartSLOduration=1.924955263 podStartE2EDuration="1.924955263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:39:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:39:58.921310134 +0000 UTC m=+1815.671731787" watchObservedRunningTime="2026-07-07 02:39:58.924955263 +0000 UTC m=+1815.675376916" Jul 07 02:39:59.153993 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:59.153962 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:59.158911 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:59.158886 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:59.871251 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:59.871221 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:39:59.872111 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:39:59.872092 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:13.948107 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:13.948074 2559 generic.go:356] "Generic (PLEG): container finished" podID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerID="bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24" exitCode=0 Jul 07 02:40:13.948577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:13.948145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" event={"ID":"2fafd300-bb71-4e76-b8e1-ec00d9f3c696","Type":"ContainerDied","Data":"bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24"} Jul 07 02:40:13.948577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:13.948560 2559 scope.go:117] "RemoveContainer" containerID="bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24" Jul 07 02:40:13.949637 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:13.949610 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:13.949696 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:13.949676 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:13.949733 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:13.949707 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:14.871376 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:14.871332 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:14.960844 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:14.960800 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" event={"ID":"2fafd300-bb71-4e76-b8e1-ec00d9f3c696","Type":"ContainerStarted","Data":"aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925"} Jul 07 02:40:15.153788 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:15.153752 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:15.155471 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:15.155386 2559 patch_prober.go:28] interesting pod/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 02:40:15.155676 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:15.155654 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 02:40:16.153768 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:16.153715 2559 patch_prober.go:28] interesting pod/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 02:40:16.154317 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:16.153792 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 02:40:17.154017 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:17.153979 2559 patch_prober.go:28] interesting pod/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 02:40:17.154705 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:17.154670 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 02:40:18.154003 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:18.153963 2559 patch_prober.go:28] interesting pod/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 07 02:40:18.154307 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:18.154277 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 07 02:40:19.157957 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:19.157929 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:19.158351 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:19.158221 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:19.158941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:19.158919 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:19.278646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:19.278614 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d"] Jul 07 02:40:19.989502 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:19.989467 2559 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" secret="" err="secret \"e2e-mt-3a3e4d5a-openshift-default-dockercfg-hh88f\" not found" Jul 07 02:40:20.996830 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:20.996791 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" containerID="cri-o://aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925" gracePeriod=30 Jul 07 02:40:26.249988 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.249961 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:26.253514 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.253494 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-podinfo\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.253634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.253550 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-envoy\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.253634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.253614 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-data\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.253742 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.253642 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-token\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.254341 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.254145 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-certs\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.254341 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.254237 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istiod-ca-cert\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.254341 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.254265 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-credential-socket\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.254341 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.254304 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vqvc\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-kube-api-access-4vqvc\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.254574 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.254356 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-socket\") pod \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\" (UID: \"2fafd300-bb71-4e76-b8e1-ec00d9f3c696\") " Jul 07 02:40:26.254715 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.254665 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:40:26.255272 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.254790 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:40:26.255272 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.255107 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:40:26.255272 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.255211 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-data" (OuterVolumeSpecName: "istio-data") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:40:26.257267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.255509 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:40:26.257267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.256280 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.257267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.256303 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.257267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.256316 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.257267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.256334 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.257267 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.256347 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.263230 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.263202 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:40:26.264425 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.264396 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-token" (OuterVolumeSpecName: "istio-token") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:40:26.264524 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.264444 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-kube-api-access-4vqvc" (OuterVolumeSpecName: "kube-api-access-4vqvc") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "kube-api-access-4vqvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:40:26.264682 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.264657 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2fafd300-bb71-4e76-b8e1-ec00d9f3c696" (UID: "2fafd300-bb71-4e76-b8e1-ec00d9f3c696"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:40:26.357634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.357588 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.357634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.357627 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.357634 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.357636 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:26.357872 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:26.357645 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vqvc\" (UniqueName: \"kubernetes.io/projected/2fafd300-bb71-4e76-b8e1-ec00d9f3c696-kube-api-access-4vqvc\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:40:27.030464 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.030431 2559 generic.go:356] "Generic (PLEG): container finished" podID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerID="aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925" exitCode=0 Jul 07 02:40:27.030671 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.030495 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" Jul 07 02:40:27.030671 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.030519 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" event={"ID":"2fafd300-bb71-4e76-b8e1-ec00d9f3c696","Type":"ContainerDied","Data":"aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925"} Jul 07 02:40:27.030671 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.030587 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d" event={"ID":"2fafd300-bb71-4e76-b8e1-ec00d9f3c696","Type":"ContainerDied","Data":"f111cda67c6823aa9ea0cf92e86ae5d8e9e13dca2d71d24e5eddcbb79779430d"} Jul 07 02:40:27.030671 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.030619 2559 scope.go:117] "RemoveContainer" containerID="aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925" Jul 07 02:40:27.040726 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.040709 2559 scope.go:117] "RemoveContainer" containerID="bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24" Jul 07 02:40:27.049052 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.049033 2559 scope.go:117] "RemoveContainer" containerID="aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925" Jul 07 02:40:27.049344 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:40:27.049323 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925\": container with ID starting with aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925 not found: ID does not exist" containerID="aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925" Jul 07 02:40:27.049409 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.049354 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925"} err="failed to get container status \"aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925\": rpc error: code = NotFound desc = could not find container \"aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925\": container with ID starting with aff411f15fbfab7e87766f57e89383a6210b9391571118ec89ad773a89b4b925 not found: ID does not exist" Jul 07 02:40:27.049409 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.049373 2559 scope.go:117] "RemoveContainer" containerID="bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24" Jul 07 02:40:27.049634 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:40:27.049619 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24\": container with ID starting with bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24 not found: ID does not exist" containerID="bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24" Jul 07 02:40:27.049684 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.049640 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24"} err="failed to get container status \"bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24\": rpc error: code = NotFound desc = could not find container \"bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24\": container with ID starting with bd80b0e3816e8fd828bc7920465644d3d235d3479d9fffc8e929e399810d7b24 not found: ID does not exist" Jul 07 02:40:27.053996 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.053970 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d"] Jul 07 02:40:27.058417 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.058389 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3a3e4d5a-openshift-default-6647478f99-8hs2d"] Jul 07 02:40:27.818889 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:27.818857 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" path="/var/lib/kubelet/pods/2fafd300-bb71-4e76-b8e1-ec00d9f3c696/volumes" Jul 07 02:40:38.814415 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.814383 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95"] Jul 07 02:40:38.814993 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.814967 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" Jul 07 02:40:38.814993 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.814992 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" Jul 07 02:40:38.815213 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.815132 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" Jul 07 02:40:38.818703 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.818666 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.821632 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.821612 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-717b0bf6-openshift-default-dockercfg-nj4dp\"" Jul 07 02:40:38.829141 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.829117 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95"] Jul 07 02:40:38.877710 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-socket\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.877895 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877753 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46ldn\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-kube-api-access-46ldn\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.877895 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877805 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-certs\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.877895 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877843 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istiod-ca-cert\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.878075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877898 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-data\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.878075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877930 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-token\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.878075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877966 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-envoy\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.878075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.877993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-podinfo\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.878075 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.878035 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-credential-socket\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979129 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979096 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-socket\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979338 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979166 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46ldn\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-kube-api-access-46ldn\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979338 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979204 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-certs\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979338 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979228 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istiod-ca-cert\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979338 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979278 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-data\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979404 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-token\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979462 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-envoy\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979492 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-podinfo\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979535 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-credential-socket\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979745 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979588 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-socket\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979745 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979635 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-certs\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979861 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979791 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-data\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.979989 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.979960 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-credential-socket\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.980279 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.980253 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istiod-ca-cert\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.981809 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.981786 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-envoy\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.982017 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.981998 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-podinfo\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.990018 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.989996 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46ldn\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-kube-api-access-46ldn\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:38.993712 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:38.993686 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-token\") pod \"e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:39.130013 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:39.129932 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:39.277974 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:39.277931 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95"] Jul 07 02:40:39.279215 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:40:39.279187 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b70bd3_9c6e_44a4_b733_e1082eacfa4d.slice/crio-4b9cfd9a2a27b38bc1438cdefaa039eb0c79963369f2b57edbedd8633f53ee36 WatchSource:0}: Error finding container 4b9cfd9a2a27b38bc1438cdefaa039eb0c79963369f2b57edbedd8633f53ee36: Status 404 returned error can't find the container with id 4b9cfd9a2a27b38bc1438cdefaa039eb0c79963369f2b57edbedd8633f53ee36 Jul 07 02:40:39.281370 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:39.281331 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:39.281465 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:39.281397 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:39.281465 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:39.281428 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:40.107033 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:40.106997 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" event={"ID":"58b70bd3-9c6e-44a4-b733-e1082eacfa4d","Type":"ContainerStarted","Data":"a2bc6ed12345cd7ac8e175e3d1b2816fcebeb5fd9ea01fbc6cae2df77749d09c"} Jul 07 02:40:40.107457 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:40.107050 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" event={"ID":"58b70bd3-9c6e-44a4-b733-e1082eacfa4d","Type":"ContainerStarted","Data":"4b9cfd9a2a27b38bc1438cdefaa039eb0c79963369f2b57edbedd8633f53ee36"} Jul 07 02:40:40.130539 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:40.130511 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:40.131740 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:40.131693 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" podStartSLOduration=2.131679102 podStartE2EDuration="2.131679102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:40:40.12883659 +0000 UTC m=+1856.879258255" watchObservedRunningTime="2026-07-07 02:40:40.131679102 +0000 UTC m=+1856.882100753" Jul 07 02:40:40.135530 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:40.135508 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:41.112282 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:41.112258 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:41.113265 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:41.113241 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:56.113666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.113587 2559 patch_prober.go:28] interesting pod/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 02:40:56.113666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.113638 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 02:40:56.204284 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.204196 2559 generic.go:356] "Generic (PLEG): container finished" podID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerID="a2bc6ed12345cd7ac8e175e3d1b2816fcebeb5fd9ea01fbc6cae2df77749d09c" exitCode=0 Jul 07 02:40:56.204284 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.204251 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" event={"ID":"58b70bd3-9c6e-44a4-b733-e1082eacfa4d","Type":"ContainerDied","Data":"a2bc6ed12345cd7ac8e175e3d1b2816fcebeb5fd9ea01fbc6cae2df77749d09c"} Jul 07 02:40:56.204667 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.204653 2559 scope.go:117] "RemoveContainer" containerID="a2bc6ed12345cd7ac8e175e3d1b2816fcebeb5fd9ea01fbc6cae2df77749d09c" Jul 07 02:40:56.205703 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.205668 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:56.205771 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.205750 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:56.205806 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:56.205796 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:40:57.212330 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:57.212292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" event={"ID":"58b70bd3-9c6e-44a4-b733-e1082eacfa4d","Type":"ContainerStarted","Data":"686445e4fa649d204f735ba0b25a50c31432156f49b285df02c5a848ac092913"} Jul 07 02:40:58.130372 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:58.130327 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:40:58.130559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:58.130535 2559 patch_prober.go:28] interesting pod/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 02:40:58.130612 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:58.130579 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 02:40:59.130978 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:59.130943 2559 patch_prober.go:28] interesting pod/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 02:40:59.131415 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:40:59.131006 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 02:41:00.130799 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:41:00.130761 2559 patch_prober.go:28] interesting pod/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 07 02:41:00.130991 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:41:00.130845 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 07 02:41:01.134255 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:41:01.134227 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:41:01.134658 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:41:01.134537 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:41:01.135097 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:41:01.135077 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:42:18.481497 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:18.481457 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95"] Jul 07 02:42:18.482082 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:18.481723 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" containerID="cri-o://686445e4fa649d204f735ba0b25a50c31432156f49b285df02c5a848ac092913" gracePeriod=30 Jul 07 02:42:23.707345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.707308 2559 generic.go:356] "Generic (PLEG): container finished" podID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerID="686445e4fa649d204f735ba0b25a50c31432156f49b285df02c5a848ac092913" exitCode=0 Jul 07 02:42:23.707727 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.707375 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" event={"ID":"58b70bd3-9c6e-44a4-b733-e1082eacfa4d","Type":"ContainerDied","Data":"686445e4fa649d204f735ba0b25a50c31432156f49b285df02c5a848ac092913"} Jul 07 02:42:23.707727 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.707438 2559 scope.go:117] "RemoveContainer" containerID="a2bc6ed12345cd7ac8e175e3d1b2816fcebeb5fd9ea01fbc6cae2df77749d09c" Jul 07 02:42:23.744929 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.744875 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:42:23.777140 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777108 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-envoy\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777174 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-socket\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777206 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-token\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777237 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-data\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777275 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46ldn\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-kube-api-access-46ldn\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777565 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777479 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-podinfo\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777565 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777502 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:42:23.777674 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777566 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-certs\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777674 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777620 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-credential-socket\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777674 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777664 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istiod-ca-cert\") pod \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\" (UID: \"58b70bd3-9c6e-44a4-b733-e1082eacfa4d\") " Jul 07 02:42:23.777993 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.777975 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.778397 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.778360 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:42:23.778397 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.778374 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:42:23.778543 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.778446 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:42:23.778756 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.778731 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:42:23.779861 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.779818 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:42:23.779861 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.779834 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:42:23.780536 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.780503 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:42:23.780628 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.780584 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-kube-api-access-46ldn" (OuterVolumeSpecName: "kube-api-access-46ldn") pod "58b70bd3-9c6e-44a4-b733-e1082eacfa4d" (UID: "58b70bd3-9c6e-44a4-b733-e1082eacfa4d"). InnerVolumeSpecName "kube-api-access-46ldn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:42:23.878785 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878751 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.878785 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878781 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.878785 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878790 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46ldn\" (UniqueName: \"kubernetes.io/projected/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-kube-api-access-46ldn\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.879115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878800 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.879115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878809 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.879115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878816 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.879115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878824 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:23.879115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:23.878832 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58b70bd3-9c6e-44a4-b733-e1082eacfa4d-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:42:24.714787 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:24.714751 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" Jul 07 02:42:24.714787 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:24.714773 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95" event={"ID":"58b70bd3-9c6e-44a4-b733-e1082eacfa4d","Type":"ContainerDied","Data":"4b9cfd9a2a27b38bc1438cdefaa039eb0c79963369f2b57edbedd8633f53ee36"} Jul 07 02:42:24.715305 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:24.714830 2559 scope.go:117] "RemoveContainer" containerID="686445e4fa649d204f735ba0b25a50c31432156f49b285df02c5a848ac092913" Jul 07 02:42:24.734118 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:24.734075 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95"] Jul 07 02:42:24.737573 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:24.737546 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-717b0bf6-openshift-default-67f544c8ff-4fd95"] Jul 07 02:42:25.819107 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:42:25.819076 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" path="/var/lib/kubelet/pods/58b70bd3-9c6e-44a4-b733-e1082eacfa4d/volumes" Jul 07 02:43:13.044487 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044453 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l"] Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044830 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044841 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044854 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044862 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044869 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044874 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044941 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" Jul 07 02:43:13.044947 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044950 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b70bd3-9c6e-44a4-b733-e1082eacfa4d" containerName="istio-proxy" Jul 07 02:43:13.045244 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.044958 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fafd300-bb71-4e76-b8e1-ec00d9f3c696" containerName="istio-proxy" Jul 07 02:43:13.047249 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.047232 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.049763 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.049743 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8f3d77-openshift-default-dockercfg-82ljz\"" Jul 07 02:43:13.060030 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.060005 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l"] Jul 07 02:43:13.118309 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-envoy\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118317 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptdps\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-kube-api-access-ptdps\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118343 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istiod-ca-cert\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118425 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-certs\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118463 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-data\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118486 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118482 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-podinfo\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118507 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-token\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118558 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-socket\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.118686 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.118584 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-credential-socket\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219140 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-envoy\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptdps\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-kube-api-access-ptdps\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219249 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istiod-ca-cert\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219283 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-certs\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219315 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-data\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219344 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-podinfo\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219393 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-token\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-socket\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219454 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-credential-socket\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219946 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219715 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-certs\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219946 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219751 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-data\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.219946 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219829 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-credential-socket\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.220104 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.219974 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-socket\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.220104 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.220017 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istiod-ca-cert\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.221616 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.221596 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-envoy\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.221745 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.221727 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-podinfo\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.228010 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.227987 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-token\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.228010 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.228007 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptdps\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-kube-api-access-ptdps\") pod \"e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.357611 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.357519 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:13.503011 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.502985 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l"] Jul 07 02:43:13.505099 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:43:13.505075 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3cf90c_1dc8_453b_af6e_4487cc7c107a.slice/crio-c1d1c6d19d82a39ecb5a1c671fe524a4fee52555c98cee5eaf951e7f91288241 WatchSource:0}: Error finding container c1d1c6d19d82a39ecb5a1c671fe524a4fee52555c98cee5eaf951e7f91288241: Status 404 returned error can't find the container with id c1d1c6d19d82a39ecb5a1c671fe524a4fee52555c98cee5eaf951e7f91288241 Jul 07 02:43:13.507754 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.507723 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:13.507861 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.507802 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:13.507861 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.507847 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:13.976965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.976930 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" event={"ID":"9c3cf90c-1dc8-453b-af6e-4487cc7c107a","Type":"ContainerStarted","Data":"e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef"} Jul 07 02:43:13.977130 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.976986 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" event={"ID":"9c3cf90c-1dc8-453b-af6e-4487cc7c107a","Type":"ContainerStarted","Data":"c1d1c6d19d82a39ecb5a1c671fe524a4fee52555c98cee5eaf951e7f91288241"} Jul 07 02:43:13.999638 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:13.999583 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" podStartSLOduration=0.999563875 podStartE2EDuration="999.563875ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:43:13.997685951 +0000 UTC m=+2010.748107627" watchObservedRunningTime="2026-07-07 02:43:13.999563875 +0000 UTC m=+2010.749985529" Jul 07 02:43:14.357982 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:14.357901 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:14.362905 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:14.362882 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:14.982574 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:14.982543 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:14.983655 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:14.983633 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:21.003171 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.003128 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f"] Jul 07 02:43:21.006508 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.006474 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.009088 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.009057 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-902459-openshift-default-dockercfg-sj2rc\"" Jul 07 02:43:21.021389 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.021365 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f"] Jul 07 02:43:21.091019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.090979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kk77\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-kube-api-access-7kk77\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091254 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091029 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-socket\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091254 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091187 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istiod-ca-cert\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091394 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091267 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-certs\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091394 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091314 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-token\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091394 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091354 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-credential-socket\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091394 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-data\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091411 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-podinfo\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.091577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.091470 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-envoy\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192108 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192068 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istiod-ca-cert\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-certs\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192202 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-token\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-credential-socket\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192266 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-data\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192319 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192295 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-podinfo\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192596 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-envoy\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192596 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kk77\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-kube-api-access-7kk77\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192596 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192388 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-socket\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192633 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-certs\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192703 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-data\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192777 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192761 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-credential-socket\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192802 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-socket\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.192945 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.192929 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istiod-ca-cert\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.194679 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.194661 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-envoy\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.194993 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.194974 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-podinfo\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.200460 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.200435 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-token\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.200570 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.200526 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kk77\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-kube-api-access-7kk77\") pod \"e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.318907 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.318820 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:21.464436 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.464412 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f"] Jul 07 02:43:21.466615 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:43:21.466589 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ead014c_3b69_4a25_9f4a_cbfdd69d5b6a.slice/crio-6d3ca65ab13124720af1c9fc1aebe739c57b188a5cd16983d379d779f02d08b4 WatchSource:0}: Error finding container 6d3ca65ab13124720af1c9fc1aebe739c57b188a5cd16983d379d779f02d08b4: Status 404 returned error can't find the container with id 6d3ca65ab13124720af1c9fc1aebe739c57b188a5cd16983d379d779f02d08b4 Jul 07 02:43:21.469205 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.469149 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:21.469336 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.469259 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:21.469336 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:21.469309 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:22.021343 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:22.021305 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" event={"ID":"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a","Type":"ContainerStarted","Data":"90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e"} Jul 07 02:43:22.021730 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:22.021362 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" event={"ID":"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a","Type":"ContainerStarted","Data":"6d3ca65ab13124720af1c9fc1aebe739c57b188a5cd16983d379d779f02d08b4"} Jul 07 02:43:22.045821 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:22.045774 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" podStartSLOduration=2.045759125 podStartE2EDuration="2.045759125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:43:22.043065172 +0000 UTC m=+2018.793486827" watchObservedRunningTime="2026-07-07 02:43:22.045759125 +0000 UTC m=+2018.796180778" Jul 07 02:43:22.320010 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:22.319913 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:22.324941 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:22.324916 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:23.026702 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:23.026673 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:23.027579 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:23.027558 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:31.023406 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:31.023347 2559 patch_prober.go:28] interesting pod/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 02:43:31.023883 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:31.023460 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 02:43:32.088761 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:32.088731 2559 generic.go:356] "Generic (PLEG): container finished" podID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerID="e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef" exitCode=0 Jul 07 02:43:32.089260 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:32.088809 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" event={"ID":"9c3cf90c-1dc8-453b-af6e-4487cc7c107a","Type":"ContainerDied","Data":"e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef"} Jul 07 02:43:32.089314 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:32.089289 2559 scope.go:117] "RemoveContainer" containerID="e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef" Jul 07 02:43:32.090422 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:32.090396 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:32.090490 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:32.090459 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:32.090530 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:32.090502 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:33.097935 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:33.097896 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" event={"ID":"9c3cf90c-1dc8-453b-af6e-4487cc7c107a","Type":"ContainerStarted","Data":"259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a"} Jul 07 02:43:33.358680 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:33.358573 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:33.358876 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:33.358706 2559 patch_prober.go:28] interesting pod/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 02:43:33.358876 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:33.358752 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 02:43:34.369178 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:34.365566 2559 patch_prober.go:28] interesting pod/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 02:43:34.369178 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:34.365644 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 02:43:35.363256 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:35.358201 2559 patch_prober.go:28] interesting pod/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 02:43:35.363256 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:35.358263 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 02:43:36.359178 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:36.358803 2559 patch_prober.go:28] interesting pod/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 07 02:43:36.359178 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:36.358870 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 07 02:43:37.361683 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:37.361651 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:37.362191 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:37.362135 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:37.362928 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:37.362903 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:43:38.235922 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:38.235269 2559 patch_prober.go:28] interesting pod/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:33842->10.133.0.58:15021: read: connection reset by peer" start-of-body= Jul 07 02:43:38.235922 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:38.235327 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": read tcp 10.133.0.2:33842->10.133.0.58:15021: read: connection reset by peer" Jul 07 02:43:39.133844 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:39.133809 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerID="90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e" exitCode=0 Jul 07 02:43:39.134275 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:39.133878 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" event={"ID":"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a","Type":"ContainerDied","Data":"90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e"} Jul 07 02:43:39.134428 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:39.134415 2559 scope.go:117] "RemoveContainer" containerID="90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e" Jul 07 02:43:39.135418 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:39.135392 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:39.135494 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:39.135462 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:39.135531 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:39.135496 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:43:40.143398 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:40.143358 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" event={"ID":"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a","Type":"ContainerStarted","Data":"c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d"} Jul 07 02:43:40.319756 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:40.319724 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:40.319957 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:40.319937 2559 patch_prober.go:28] interesting pod/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 02:43:40.320010 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:40.319986 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 02:43:41.319545 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:41.319506 2559 patch_prober.go:28] interesting pod/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 02:43:41.319924 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:41.319585 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 02:43:42.320081 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:42.320046 2559 patch_prober.go:28] interesting pod/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 02:43:42.320492 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:42.320111 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 02:43:43.320104 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:43.320068 2559 patch_prober.go:28] interesting pod/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 02:43:43.320588 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:43.320136 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 02:43:44.323097 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:44.323066 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:44.323567 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:44.323440 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:43:44.324390 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:43:44.324372 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:45:10.278264 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:10.278230 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l"] Jul 07 02:45:10.278700 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:10.278440 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" containerID="cri-o://259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a" gracePeriod=30 Jul 07 02:45:15.536396 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.536362 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:45:15.610754 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610674 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-envoy\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.610754 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610709 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-credential-socket\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.610754 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610752 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-token\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.611032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610772 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-socket\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.611032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610801 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-certs\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.611032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610821 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-data\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.611032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610838 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-podinfo\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.611032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610902 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptdps\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-kube-api-access-ptdps\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.611032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.610946 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istiod-ca-cert\") pod \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\" (UID: \"9c3cf90c-1dc8-453b-af6e-4487cc7c107a\") " Jul 07 02:45:15.611381 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.611138 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:15.611381 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.611201 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:15.611490 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.611382 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.611490 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.611402 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.611490 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.611447 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:15.611490 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.611464 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:45:15.611931 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.611899 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:15.612878 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.612856 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:15.613419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.613389 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:45:15.613419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.613414 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:45:15.613798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.613780 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-kube-api-access-ptdps" (OuterVolumeSpecName: "kube-api-access-ptdps") pod "9c3cf90c-1dc8-453b-af6e-4487cc7c107a" (UID: "9c3cf90c-1dc8-453b-af6e-4487cc7c107a"). InnerVolumeSpecName "kube-api-access-ptdps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:45:15.703399 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.703367 2559 generic.go:356] "Generic (PLEG): container finished" podID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerID="259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a" exitCode=0 Jul 07 02:45:15.703577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.703420 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" Jul 07 02:45:15.703577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.703450 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" event={"ID":"9c3cf90c-1dc8-453b-af6e-4487cc7c107a","Type":"ContainerDied","Data":"259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a"} Jul 07 02:45:15.703577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.703504 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l" event={"ID":"9c3cf90c-1dc8-453b-af6e-4487cc7c107a","Type":"ContainerDied","Data":"c1d1c6d19d82a39ecb5a1c671fe524a4fee52555c98cee5eaf951e7f91288241"} Jul 07 02:45:15.703577 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.703534 2559 scope.go:117] "RemoveContainer" containerID="259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a" Jul 07 02:45:15.711907 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.711884 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptdps\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-kube-api-access-ptdps\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.712019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.711909 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.712019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.711923 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.712019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.711935 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.712019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.711946 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.712019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.711958 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.712019 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.711968 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3cf90c-1dc8-453b-af6e-4487cc7c107a-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:15.713495 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.713479 2559 scope.go:117] "RemoveContainer" containerID="e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef" Jul 07 02:45:15.722101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.722084 2559 scope.go:117] "RemoveContainer" containerID="259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a" Jul 07 02:45:15.722410 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:45:15.722387 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a\": container with ID starting with 259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a not found: ID does not exist" containerID="259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a" Jul 07 02:45:15.722516 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.722422 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a"} err="failed to get container status \"259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a\": rpc error: code = NotFound desc = could not find container \"259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a\": container with ID starting with 259868b9374f2dfcf876e2931fffd79eebf643e5eddeb3dcd925c79ee06db25a not found: ID does not exist" Jul 07 02:45:15.722516 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.722445 2559 scope.go:117] "RemoveContainer" containerID="e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef" Jul 07 02:45:15.722726 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:45:15.722706 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef\": container with ID starting with e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef not found: ID does not exist" containerID="e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef" Jul 07 02:45:15.722788 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.722733 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef"} err="failed to get container status \"e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef\": rpc error: code = NotFound desc = could not find container \"e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef\": container with ID starting with e13fb02934da4a2339cde7da957a0390edf1f7e4837a72df203f91eb51b05fef not found: ID does not exist" Jul 07 02:45:15.727884 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.727863 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l"] Jul 07 02:45:15.731305 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.731281 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8f3d77-openshift-default-66bc559947-wh96l"] Jul 07 02:45:15.818494 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:15.818465 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" path="/var/lib/kubelet/pods/9c3cf90c-1dc8-453b-af6e-4487cc7c107a/volumes" Jul 07 02:45:17.245933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:17.245891 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f"] Jul 07 02:45:17.246410 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:17.246195 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" containerID="cri-o://c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d" gracePeriod=30 Jul 07 02:45:18.100870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.100832 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92"] Jul 07 02:45:18.101273 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.101258 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" Jul 07 02:45:18.101273 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.101273 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" Jul 07 02:45:18.101401 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.101286 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" Jul 07 02:45:18.101401 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.101292 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" Jul 07 02:45:18.101401 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.101351 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" Jul 07 02:45:18.101401 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.101362 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3cf90c-1dc8-453b-af6e-4487cc7c107a" containerName="istio-proxy" Jul 07 02:45:18.103731 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.103707 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.106239 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.106216 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8ad891-openshift-default-dockercfg-b4whs\"" Jul 07 02:45:18.115371 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.115345 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92"] Jul 07 02:45:18.133070 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133035 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-credential-socket\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133262 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133103 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-certs\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133262 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133141 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abadb4c-293f-44a3-963b-a70264abee5c-istio-podinfo\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133262 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133192 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2zrr\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-kube-api-access-c2zrr\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133444 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133250 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abadb4c-293f-44a3-963b-a70264abee5c-istiod-ca-cert\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133444 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133309 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-istio-token\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133444 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-envoy\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133447 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-data\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.133589 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.133492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-socket\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234585 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-istio-token\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234772 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234620 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-envoy\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234772 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234657 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-data\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234772 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234683 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-socket\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234772 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234734 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-credential-socket\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234951 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234780 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-certs\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234951 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234813 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abadb4c-293f-44a3-963b-a70264abee5c-istio-podinfo\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234951 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234837 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2zrr\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-kube-api-access-c2zrr\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.234951 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.234862 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abadb4c-293f-44a3-963b-a70264abee5c-istiod-ca-cert\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.235287 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.235236 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-data\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.235287 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.235245 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-socket\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.235493 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.235413 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-credential-socket\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.235493 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.235434 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-certs\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.235697 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.235674 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abadb4c-293f-44a3-963b-a70264abee5c-istiod-ca-cert\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.237106 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.237086 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-envoy\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.237473 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.237454 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abadb4c-293f-44a3-963b-a70264abee5c-istio-podinfo\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.242786 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.242766 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-istio-token\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.242926 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.242904 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2zrr\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-kube-api-access-c2zrr\") pod \"e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.415021 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.414987 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:18.559900 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.559867 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92"] Jul 07 02:45:18.560784 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:45:18.560758 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8abadb4c_293f_44a3_963b_a70264abee5c.slice/crio-9f1444af488497ed3409fdfb5329a958dcb8fffcfd34c68254d7041f086f9346 WatchSource:0}: Error finding container 9f1444af488497ed3409fdfb5329a958dcb8fffcfd34c68254d7041f086f9346: Status 404 returned error can't find the container with id 9f1444af488497ed3409fdfb5329a958dcb8fffcfd34c68254d7041f086f9346 Jul 07 02:45:18.563048 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.563011 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:18.563144 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.563080 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:18.563144 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.563109 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:18.723812 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.723706 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" event={"ID":"8abadb4c-293f-44a3-963b-a70264abee5c","Type":"ContainerStarted","Data":"58a6c8021c94b1b57436c12192b1e6f75ded8c19c038b46a64091606b584ff36"} Jul 07 02:45:18.723812 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.723776 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" event={"ID":"8abadb4c-293f-44a3-963b-a70264abee5c","Type":"ContainerStarted","Data":"9f1444af488497ed3409fdfb5329a958dcb8fffcfd34c68254d7041f086f9346"} Jul 07 02:45:18.745764 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:18.745704 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" podStartSLOduration=0.745684632 podStartE2EDuration="745.684632ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:45:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:45:18.742626676 +0000 UTC m=+2135.493048329" watchObservedRunningTime="2026-07-07 02:45:18.745684632 +0000 UTC m=+2135.496106287" Jul 07 02:45:19.415195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:19.415135 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:19.420299 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:19.420270 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:19.729550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:19.729470 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:19.730400 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:19.730380 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:22.504036 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.504009 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:45:22.575477 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575390 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-credential-socket\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.575477 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575437 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istiod-ca-cert\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.575704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575522 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-envoy\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.575704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575550 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kk77\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-kube-api-access-7kk77\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.575704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575585 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-podinfo\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.575704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575610 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-token\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.575704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575651 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-certs\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.575704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575671 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:22.575704 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575686 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-data\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.576071 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575964 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:22.576071 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.575987 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:45:22.576071 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.576056 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-socket\") pod \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\" (UID: \"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a\") " Jul 07 02:45:22.576484 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.576461 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:22.576568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.576468 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.576568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.576513 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.576568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.576529 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.576568 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.576525 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:22.577835 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.577814 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:45:22.577913 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.577898 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:22.578129 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.578111 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-kube-api-access-7kk77" (OuterVolumeSpecName: "kube-api-access-7kk77") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "kube-api-access-7kk77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:45:22.578192 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.578146 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" (UID: "9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:45:22.677349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.677310 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.677349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.677343 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kk77\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-kube-api-access-7kk77\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.677349 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.677355 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.677586 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.677363 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.677586 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.677371 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.677586 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.677378 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:22.747962 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.747929 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerID="c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d" exitCode=0 Jul 07 02:45:22.748196 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.747970 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" event={"ID":"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a","Type":"ContainerDied","Data":"c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d"} Jul 07 02:45:22.748196 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.747987 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" Jul 07 02:45:22.748196 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.748014 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f" event={"ID":"9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a","Type":"ContainerDied","Data":"6d3ca65ab13124720af1c9fc1aebe739c57b188a5cd16983d379d779f02d08b4"} Jul 07 02:45:22.748196 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.748037 2559 scope.go:117] "RemoveContainer" containerID="c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d" Jul 07 02:45:22.757972 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.757951 2559 scope.go:117] "RemoveContainer" containerID="90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e" Jul 07 02:45:22.766144 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.766125 2559 scope.go:117] "RemoveContainer" containerID="c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d" Jul 07 02:45:22.766466 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:45:22.766441 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d\": container with ID starting with c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d not found: ID does not exist" containerID="c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d" Jul 07 02:45:22.766509 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.766479 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d"} err="failed to get container status \"c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d\": rpc error: code = NotFound desc = could not find container \"c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d\": container with ID starting with c6c26546bfcac2190ab4e6a92691e62d76a41eeca2cca1258c0ea40f95e4392d not found: ID does not exist" Jul 07 02:45:22.766509 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.766500 2559 scope.go:117] "RemoveContainer" containerID="90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e" Jul 07 02:45:22.766847 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:45:22.766804 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e\": container with ID starting with 90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e not found: ID does not exist" containerID="90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e" Jul 07 02:45:22.767188 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.766840 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e"} err="failed to get container status \"90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e\": rpc error: code = NotFound desc = could not find container \"90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e\": container with ID starting with 90286f8e6248cd9bd6019440b55c4e8aa4af740ee8826be2a0def369d1ab707e not found: ID does not exist" Jul 07 02:45:22.769698 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.769677 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f"] Jul 07 02:45:22.774423 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:22.774401 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-902459-openshift-default-6fb5bdd657-9jp7f"] Jul 07 02:45:23.820788 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:23.820757 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" path="/var/lib/kubelet/pods/9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a/volumes" Jul 07 02:45:25.872808 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.872773 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc"] Jul 07 02:45:25.873315 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.873298 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" Jul 07 02:45:25.873315 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.873316 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" Jul 07 02:45:25.873409 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.873339 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" Jul 07 02:45:25.873409 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.873345 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" Jul 07 02:45:25.873409 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.873402 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" Jul 07 02:45:25.873502 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.873413 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ead014c-3b69-4a25-9f4a-cbfdd69d5b6a" containerName="istio-proxy" Jul 07 02:45:25.877188 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.877168 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.879915 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.879874 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-833817-openshift-default-dockercfg-jq7xl\"" Jul 07 02:45:25.889064 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.889037 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc"] Jul 07 02:45:25.909252 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-credential-socket\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909418 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909280 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-certs\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909418 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909391 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-data\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909533 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fcdd2ac-082c-4653-be56-348f9822242e-istio-podinfo\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909591 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909555 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fcdd2ac-082c-4653-be56-348f9822242e-istiod-ca-cert\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-envoy\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909701 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-socket\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909759 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv5g2\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-kube-api-access-pv5g2\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:25.909808 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:25.909767 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-istio-token\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011231 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011183 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-data\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011429 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011250 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fcdd2ac-082c-4653-be56-348f9822242e-istio-podinfo\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011429 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011277 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fcdd2ac-082c-4653-be56-348f9822242e-istiod-ca-cert\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011429 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-envoy\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-socket\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv5g2\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-kube-api-access-pv5g2\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-istio-token\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011561 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-data\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011603 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011583 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-credential-socket\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011867 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011621 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-certs\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011867 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011770 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-socket\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.011986 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011914 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-credential-socket\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.012044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.011992 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-certs\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.012105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.012081 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fcdd2ac-082c-4653-be56-348f9822242e-istiod-ca-cert\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.013678 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.013660 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-envoy\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.013820 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.013803 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fcdd2ac-082c-4653-be56-348f9822242e-istio-podinfo\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.020468 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.020444 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-istio-token\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.020651 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.020634 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv5g2\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-kube-api-access-pv5g2\") pod \"e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.190545 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.190508 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:26.344451 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.344422 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc"] Jul 07 02:45:26.345413 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:45:26.345387 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fcdd2ac_082c_4653_be56_348f9822242e.slice/crio-f539d56d434988f3b630dab46add7af6368609715016ded6e08599c3d9a932f3 WatchSource:0}: Error finding container f539d56d434988f3b630dab46add7af6368609715016ded6e08599c3d9a932f3: Status 404 returned error can't find the container with id f539d56d434988f3b630dab46add7af6368609715016ded6e08599c3d9a932f3 Jul 07 02:45:26.347695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.347651 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:26.347790 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.347744 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:26.347833 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.347790 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:26.774123 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.774089 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" event={"ID":"9fcdd2ac-082c-4653-be56-348f9822242e","Type":"ContainerStarted","Data":"f550992fd7f3ec49b6bec580bd8a54fca39979c94840219ebea59e65ed567ce3"} Jul 07 02:45:26.774350 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.774141 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" event={"ID":"9fcdd2ac-082c-4653-be56-348f9822242e","Type":"ContainerStarted","Data":"f539d56d434988f3b630dab46add7af6368609715016ded6e08599c3d9a932f3"} Jul 07 02:45:26.795400 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:26.795349 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" podStartSLOduration=1.795333489 podStartE2EDuration="1.795333489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:45:26.792881977 +0000 UTC m=+2143.543303667" watchObservedRunningTime="2026-07-07 02:45:26.795333489 +0000 UTC m=+2143.545755141" Jul 07 02:45:27.191594 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:27.191556 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:27.196619 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:27.196595 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:27.780309 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:27.780274 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:27.781286 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:27.781267 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:35.771379 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:35.771329 2559 patch_prober.go:28] interesting pod/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 02:45:35.773252 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:35.771396 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 02:45:36.834062 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:36.834026 2559 generic.go:356] "Generic (PLEG): container finished" podID="8abadb4c-293f-44a3-963b-a70264abee5c" containerID="58a6c8021c94b1b57436c12192b1e6f75ded8c19c038b46a64091606b584ff36" exitCode=0 Jul 07 02:45:36.834477 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:36.834094 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" event={"ID":"8abadb4c-293f-44a3-963b-a70264abee5c","Type":"ContainerDied","Data":"58a6c8021c94b1b57436c12192b1e6f75ded8c19c038b46a64091606b584ff36"} Jul 07 02:45:36.834523 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:36.834485 2559 scope.go:117] "RemoveContainer" containerID="58a6c8021c94b1b57436c12192b1e6f75ded8c19c038b46a64091606b584ff36" Jul 07 02:45:36.835516 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:36.835489 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:36.835584 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:36.835554 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:36.835625 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:36.835582 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:37.843563 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:37.842546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" event={"ID":"8abadb4c-293f-44a3-963b-a70264abee5c","Type":"ContainerStarted","Data":"2ea14d1f5e33a2409effa30258bc4296c09444db74ee8d0800b06d0bc1180e75"} Jul 07 02:45:38.416192 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:38.416137 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:38.416373 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:38.416278 2559 patch_prober.go:28] interesting pod/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 02:45:38.416373 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:38.416328 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 02:45:39.416192 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:39.416139 2559 patch_prober.go:28] interesting pod/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 02:45:39.416582 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:39.416222 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 02:45:40.415948 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:40.415904 2559 patch_prober.go:28] interesting pod/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 02:45:40.416208 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:40.415977 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 02:45:41.415643 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:41.415608 2559 patch_prober.go:28] interesting pod/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 02:45:41.415830 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:41.415673 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 02:45:42.419846 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.419816 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:42.420253 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.420133 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:42.420796 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.420776 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:42.782586 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.782487 2559 patch_prober.go:28] interesting pod/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 02:45:42.782586 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.782541 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 02:45:42.891221 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.891189 2559 generic.go:356] "Generic (PLEG): container finished" podID="9fcdd2ac-082c-4653-be56-348f9822242e" containerID="f550992fd7f3ec49b6bec580bd8a54fca39979c94840219ebea59e65ed567ce3" exitCode=0 Jul 07 02:45:42.891402 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.891265 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" event={"ID":"9fcdd2ac-082c-4653-be56-348f9822242e","Type":"ContainerDied","Data":"f550992fd7f3ec49b6bec580bd8a54fca39979c94840219ebea59e65ed567ce3"} Jul 07 02:45:42.891741 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.891727 2559 scope.go:117] "RemoveContainer" containerID="f550992fd7f3ec49b6bec580bd8a54fca39979c94840219ebea59e65ed567ce3" Jul 07 02:45:42.892689 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.892663 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:42.892754 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.892727 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:42.892809 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:42.892768 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:43.898397 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:43.898356 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" event={"ID":"9fcdd2ac-082c-4653-be56-348f9822242e","Type":"ContainerStarted","Data":"300dd2b3e1eae7e9c98089b11a743856996ac433e371c37976c4c65d80f267c9"} Jul 07 02:45:44.190886 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:44.190843 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:44.191128 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:44.191105 2559 patch_prober.go:28] interesting pod/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 02:45:44.191277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:44.191243 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 02:45:45.191475 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:45.191437 2559 patch_prober.go:28] interesting pod/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 02:45:45.191870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:45.191505 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 02:45:46.191620 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:46.191586 2559 patch_prober.go:28] interesting pod/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 02:45:46.192004 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:46.191639 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 02:45:47.191330 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:47.191292 2559 patch_prober.go:28] interesting pod/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 07 02:45:47.191531 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:47.191344 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 07 02:45:47.737377 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:47.737342 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92"] Jul 07 02:45:47.737778 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:47.737533 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" containerID="cri-o://2ea14d1f5e33a2409effa30258bc4296c09444db74ee8d0800b06d0bc1180e75" gracePeriod=30 Jul 07 02:45:48.195849 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:48.195816 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:48.196315 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:48.196299 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:48.197137 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:48.197116 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:45:52.952623 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:52.952591 2559 generic.go:356] "Generic (PLEG): container finished" podID="8abadb4c-293f-44a3-963b-a70264abee5c" containerID="2ea14d1f5e33a2409effa30258bc4296c09444db74ee8d0800b06d0bc1180e75" exitCode=0 Jul 07 02:45:52.953143 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:52.952661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" event={"ID":"8abadb4c-293f-44a3-963b-a70264abee5c","Type":"ContainerDied","Data":"2ea14d1f5e33a2409effa30258bc4296c09444db74ee8d0800b06d0bc1180e75"} Jul 07 02:45:52.953143 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:52.952731 2559 scope.go:117] "RemoveContainer" containerID="58a6c8021c94b1b57436c12192b1e6f75ded8c19c038b46a64091606b584ff36" Jul 07 02:45:53.019098 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.019034 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:53.106521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106489 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-credential-socket\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.106687 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106537 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-socket\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.106687 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106560 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-certs\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.106687 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106609 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2zrr\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-kube-api-access-c2zrr\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.106687 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106637 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-data\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.106872 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106724 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abadb4c-293f-44a3-963b-a70264abee5c-istiod-ca-cert\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.106872 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106760 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-envoy\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.106872 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106791 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:53.106872 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106808 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:53.106872 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106800 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abadb4c-293f-44a3-963b-a70264abee5c-istio-podinfo\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.107138 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106892 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-istio-token\") pod \"8abadb4c-293f-44a3-963b-a70264abee5c\" (UID: \"8abadb4c-293f-44a3-963b-a70264abee5c\") " Jul 07 02:45:53.107138 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.106921 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:53.107285 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.107143 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8abadb4c-293f-44a3-963b-a70264abee5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:45:53.107356 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.107339 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.107413 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.107363 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.107413 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.107377 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.107413 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.107390 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abadb4c-293f-44a3-963b-a70264abee5c-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.107533 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.107427 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:53.109392 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.109362 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8abadb4c-293f-44a3-963b-a70264abee5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:45:53.109511 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.109411 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:45:53.109511 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.109434 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-kube-api-access-c2zrr" (OuterVolumeSpecName: "kube-api-access-c2zrr") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "kube-api-access-c2zrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:45:53.109511 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.109449 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8abadb4c-293f-44a3-963b-a70264abee5c" (UID: "8abadb4c-293f-44a3-963b-a70264abee5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:45:53.208515 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.208463 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.208515 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.208513 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2zrr\" (UniqueName: \"kubernetes.io/projected/8abadb4c-293f-44a3-963b-a70264abee5c-kube-api-access-c2zrr\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.208515 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.208524 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.208515 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.208532 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abadb4c-293f-44a3-963b-a70264abee5c-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.208811 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.208541 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abadb4c-293f-44a3-963b-a70264abee5c-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:45:53.962695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.962667 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" Jul 07 02:45:53.962695 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.962681 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92" event={"ID":"8abadb4c-293f-44a3-963b-a70264abee5c","Type":"ContainerDied","Data":"9f1444af488497ed3409fdfb5329a958dcb8fffcfd34c68254d7041f086f9346"} Jul 07 02:45:53.963219 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.962738 2559 scope.go:117] "RemoveContainer" containerID="2ea14d1f5e33a2409effa30258bc4296c09444db74ee8d0800b06d0bc1180e75" Jul 07 02:45:53.984180 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.984139 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92"] Jul 07 02:45:53.989542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:53.989517 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8ad891-openshift-default-78f865ccc-bvj92"] Jul 07 02:45:54.820593 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:54.820556 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc"] Jul 07 02:45:54.820843 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:54.820813 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" containerID="cri-o://300dd2b3e1eae7e9c98089b11a743856996ac433e371c37976c4c65d80f267c9" gracePeriod=30 Jul 07 02:45:55.661625 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.661590 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p"] Jul 07 02:45:55.662011 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.661995 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" Jul 07 02:45:55.662011 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.662006 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" Jul 07 02:45:55.662101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.662016 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" Jul 07 02:45:55.662101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.662022 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" Jul 07 02:45:55.662101 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.662096 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" Jul 07 02:45:55.662223 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.662103 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" containerName="istio-proxy" Jul 07 02:45:55.665487 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.665468 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.668143 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.668122 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f01161-openshift-default-dockercfg-2s7v8\"" Jul 07 02:45:55.676049 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.676027 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p"] Jul 07 02:45:55.730194 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730141 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-certs\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730194 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730198 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-data\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730229 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-credential-socket\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730262 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-token\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730322 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-envoy\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-podinfo\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730354 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9041a6da-c8a4-4871-a381-72ccad53eea2-istiod-ca-cert\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730474 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89nz6\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-kube-api-access-89nz6\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.730735 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.730485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-socket\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.819666 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.819633 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8abadb4c-293f-44a3-963b-a70264abee5c" path="/var/lib/kubelet/pods/8abadb4c-293f-44a3-963b-a70264abee5c/volumes" Jul 07 02:45:55.831361 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-socket\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831499 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831377 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-certs\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831499 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831400 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-data\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831499 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831442 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-credential-socket\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831499 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-token\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831691 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831540 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-envoy\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831691 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831564 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-podinfo\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831691 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9041a6da-c8a4-4871-a381-72ccad53eea2-istiod-ca-cert\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831691 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831613 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89nz6\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-kube-api-access-89nz6\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831723 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-socket\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831792 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-certs\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.831906 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831863 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-credential-socket\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.832034 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.831910 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-data\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.832345 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.832326 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9041a6da-c8a4-4871-a381-72ccad53eea2-istiod-ca-cert\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.833859 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.833836 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-envoy\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.834145 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.834124 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-podinfo\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.840064 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.840040 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89nz6\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-kube-api-access-89nz6\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.840145 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.840112 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-token\") pod \"e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:55.976498 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:55.976415 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:56.125353 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:56.125319 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p"] Jul 07 02:45:56.126669 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:45:56.126636 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9041a6da_c8a4_4871_a381_72ccad53eea2.slice/crio-bc9735b09b6ab285979c768e7855e75d3724af31c576cd7daafa0c9f051d1260 WatchSource:0}: Error finding container bc9735b09b6ab285979c768e7855e75d3724af31c576cd7daafa0c9f051d1260: Status 404 returned error can't find the container with id bc9735b09b6ab285979c768e7855e75d3724af31c576cd7daafa0c9f051d1260 Jul 07 02:45:56.128990 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:56.128958 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:56.129082 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:56.129026 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:56.129082 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:56.129054 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:45:56.989917 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:56.989870 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" event={"ID":"9041a6da-c8a4-4871-a381-72ccad53eea2","Type":"ContainerStarted","Data":"a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e"} Jul 07 02:45:56.990323 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:56.989956 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" event={"ID":"9041a6da-c8a4-4871-a381-72ccad53eea2","Type":"ContainerStarted","Data":"bc9735b09b6ab285979c768e7855e75d3724af31c576cd7daafa0c9f051d1260"} Jul 07 02:45:57.010505 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:57.010442 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podStartSLOduration=2.010421507 podStartE2EDuration="2.010421507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:45:57.009867031 +0000 UTC m=+2173.760288685" watchObservedRunningTime="2026-07-07 02:45:57.010421507 +0000 UTC m=+2173.760843161" Jul 07 02:45:57.977638 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:57.977599 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:57.982642 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:57.982618 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:57.996137 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:57.996102 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:45:57.997105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:45:57.997090 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:46:00.011824 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.011791 2559 generic.go:356] "Generic (PLEG): container finished" podID="9fcdd2ac-082c-4653-be56-348f9822242e" containerID="300dd2b3e1eae7e9c98089b11a743856996ac433e371c37976c4c65d80f267c9" exitCode=0 Jul 07 02:46:00.012321 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.011863 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" event={"ID":"9fcdd2ac-082c-4653-be56-348f9822242e","Type":"ContainerDied","Data":"300dd2b3e1eae7e9c98089b11a743856996ac433e371c37976c4c65d80f267c9"} Jul 07 02:46:00.012321 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.011929 2559 scope.go:117] "RemoveContainer" containerID="f550992fd7f3ec49b6bec580bd8a54fca39979c94840219ebea59e65ed567ce3" Jul 07 02:46:00.081120 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.081045 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:46:00.170643 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170609 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-credential-socket\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.170815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170668 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fcdd2ac-082c-4653-be56-348f9822242e-istio-podinfo\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.170815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170691 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-istio-token\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.170815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170737 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-socket\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.170815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170776 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-envoy\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.170815 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170796 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv5g2\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-kube-api-access-pv5g2\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.171100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170839 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-certs\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.171100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170866 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fcdd2ac-082c-4653-be56-348f9822242e-istiod-ca-cert\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.171100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170891 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-data\") pod \"9fcdd2ac-082c-4653-be56-348f9822242e\" (UID: \"9fcdd2ac-082c-4653-be56-348f9822242e\") " Jul 07 02:46:00.171100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.170968 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:46:00.171100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.171088 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:46:00.171384 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.171245 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.171384 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.171266 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.171384 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.171276 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fcdd2ac-082c-4653-be56-348f9822242e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:46:00.171384 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.171347 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:46:00.171952 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.171922 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:46:00.173213 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.173136 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:46:00.173327 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.173287 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:46:00.173389 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.173374 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fcdd2ac-082c-4653-be56-348f9822242e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:46:00.173429 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.173405 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-kube-api-access-pv5g2" (OuterVolumeSpecName: "kube-api-access-pv5g2") pod "9fcdd2ac-082c-4653-be56-348f9822242e" (UID: "9fcdd2ac-082c-4653-be56-348f9822242e"). InnerVolumeSpecName "kube-api-access-pv5g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:46:00.272105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.272067 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fcdd2ac-082c-4653-be56-348f9822242e-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.272105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.272096 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.272105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.272105 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.272105 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.272113 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv5g2\" (UniqueName: \"kubernetes.io/projected/9fcdd2ac-082c-4653-be56-348f9822242e-kube-api-access-pv5g2\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.272396 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.272122 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.272396 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.272135 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fcdd2ac-082c-4653-be56-348f9822242e-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:00.272396 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:00.272146 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fcdd2ac-082c-4653-be56-348f9822242e-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:46:01.019044 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:01.019011 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" Jul 07 02:46:01.019500 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:01.019052 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc" event={"ID":"9fcdd2ac-082c-4653-be56-348f9822242e","Type":"ContainerDied","Data":"f539d56d434988f3b630dab46add7af6368609715016ded6e08599c3d9a932f3"} Jul 07 02:46:01.019500 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:01.019109 2559 scope.go:117] "RemoveContainer" containerID="300dd2b3e1eae7e9c98089b11a743856996ac433e371c37976c4c65d80f267c9" Jul 07 02:46:01.042480 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:01.042454 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc"] Jul 07 02:46:01.046597 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:01.046574 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-833817-openshift-default-5bfc756b45-8hgbc"] Jul 07 02:46:01.819337 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:01.819304 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" path="/var/lib/kubelet/pods/9fcdd2ac-082c-4653-be56-348f9822242e/volumes" Jul 07 02:46:04.021039 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.020999 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7"] Jul 07 02:46:04.021637 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.021612 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" Jul 07 02:46:04.021637 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.021634 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" Jul 07 02:46:04.021787 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.021745 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" Jul 07 02:46:04.025521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.025502 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.027803 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.027780 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3dabf0-openshift-default-dockercfg-zf26x\"" Jul 07 02:46:04.036949 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.036915 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7"] Jul 07 02:46:04.109715 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109679 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkl2g\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-kube-api-access-fkl2g\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.109887 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-certs\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.109887 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109748 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-envoy\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.109887 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-socket\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.109887 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-podinfo\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.109887 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-data\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.110132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-credential-socket\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.110132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109950 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istiod-ca-cert\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.110132 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.109979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-token\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211280 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-certs\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211280 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211280 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-envoy\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-socket\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-podinfo\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-data\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211397 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-credential-socket\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211468 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istiod-ca-cert\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211542 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211497 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-token\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211865 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkl2g\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-kube-api-access-fkl2g\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211865 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211704 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-certs\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.211865 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.211750 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-socket\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.212042 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.212016 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-data\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.212104 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.212054 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-credential-socket\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.212316 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.212290 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istiod-ca-cert\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.213667 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.213646 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-envoy\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.214015 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.213992 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-podinfo\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.219799 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.219766 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkl2g\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-kube-api-access-fkl2g\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.219908 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.219852 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-token\") pod \"e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.337710 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.337622 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:04.481716 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.481686 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7"] Jul 07 02:46:04.483461 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:46:04.483438 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b304a3a_b41d_4bb7_b337_3a48be717aa1.slice/crio-6203dd254176d75b664fddae98253886b6fbf3d0233e27ca9547292b2b3f46f2 WatchSource:0}: Error finding container 6203dd254176d75b664fddae98253886b6fbf3d0233e27ca9547292b2b3f46f2: Status 404 returned error can't find the container with id 6203dd254176d75b664fddae98253886b6fbf3d0233e27ca9547292b2b3f46f2 Jul 07 02:46:04.485796 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.485761 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:04.485863 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.485831 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:04.485905 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:04.485867 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:05.046466 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:05.046425 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" event={"ID":"6b304a3a-b41d-4bb7-b337-3a48be717aa1","Type":"ContainerStarted","Data":"9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff"} Jul 07 02:46:05.046827 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:05.046492 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" event={"ID":"6b304a3a-b41d-4bb7-b337-3a48be717aa1","Type":"ContainerStarted","Data":"6203dd254176d75b664fddae98253886b6fbf3d0233e27ca9547292b2b3f46f2"} Jul 07 02:46:05.068460 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:05.068411 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podStartSLOduration=2.068394469 podStartE2EDuration="2.068394469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 02:46:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:46:05.06711312 +0000 UTC m=+2181.817534774" watchObservedRunningTime="2026-07-07 02:46:05.068394469 +0000 UTC m=+2181.818816123" Jul 07 02:46:05.338426 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:05.338333 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:05.339899 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:05.339875 2559 patch_prober.go:28] interesting pod/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 02:46:05.339996 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:05.339928 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 02:46:06.343199 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:06.343145 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:07.062638 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:07.062601 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:07.065669 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:07.065637 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:14.038367 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.038322 2559 patch_prober.go:28] interesting pod/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 02:46:14.053415 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.038405 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 02:46:14.106965 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.106934 2559 generic.go:356] "Generic (PLEG): container finished" podID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerID="a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e" exitCode=0 Jul 07 02:46:14.107114 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.107006 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" event={"ID":"9041a6da-c8a4-4871-a381-72ccad53eea2","Type":"ContainerDied","Data":"a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e"} Jul 07 02:46:14.107419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.107396 2559 scope.go:117] "RemoveContainer" containerID="a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e" Jul 07 02:46:14.108399 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.108368 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:14.108461 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.108431 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:14.108498 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:14.108480 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:15.114489 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:15.114453 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" event={"ID":"9041a6da-c8a4-4871-a381-72ccad53eea2","Type":"ContainerStarted","Data":"ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17"} Jul 07 02:46:15.977070 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:15.977026 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:46:15.977318 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:15.977194 2559 patch_prober.go:28] interesting pod/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 02:46:15.977318 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:15.977240 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 02:46:16.977503 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:16.977469 2559 patch_prober.go:28] interesting pod/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 02:46:16.978110 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:16.978076 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 02:46:17.977553 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:17.977513 2559 patch_prober.go:28] interesting pod/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 02:46:17.978017 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:17.977593 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 02:46:18.977014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:18.976974 2559 patch_prober.go:28] interesting pod/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 07 02:46:18.977235 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:18.977037 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 07 02:46:19.980605 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:19.980573 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:46:19.980982 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:19.980961 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:46:19.981748 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:19.981730 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:46:21.149092 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:21.148999 2559 generic.go:356] "Generic (PLEG): container finished" podID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerID="9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff" exitCode=0 Jul 07 02:46:21.149092 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:21.149075 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" event={"ID":"6b304a3a-b41d-4bb7-b337-3a48be717aa1","Type":"ContainerDied","Data":"9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff"} Jul 07 02:46:21.149681 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:21.149644 2559 scope.go:117] "RemoveContainer" containerID="9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff" Jul 07 02:46:21.150675 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:21.150644 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:21.150806 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:21.150726 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:21.150806 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:21.150771 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 02:46:22.062754 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:22.062722 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:22.156441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:22.156407 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" event={"ID":"6b304a3a-b41d-4bb7-b337-3a48be717aa1","Type":"ContainerStarted","Data":"8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287"} Jul 07 02:46:22.338644 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:22.338522 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:22.339679 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:22.339637 2559 patch_prober.go:28] interesting pod/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 02:46:22.339863 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:22.339741 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 02:46:23.338754 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:23.338708 2559 patch_prober.go:28] interesting pod/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 02:46:23.339320 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:23.338776 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 02:46:24.338987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:24.338945 2559 patch_prober.go:28] interesting pod/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 02:46:24.339721 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:24.339682 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 02:46:25.338494 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:25.338455 2559 patch_prober.go:28] interesting pod/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 02:46:25.338781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:25.338750 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 02:46:26.338240 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:26.338143 2559 patch_prober.go:28] interesting pod/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 02:46:26.339259 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:26.338812 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 02:46:27.342600 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:27.342564 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:27.343048 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:27.342844 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:46:27.343436 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:46:27.343419 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:52:18.127199 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:18.127146 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p"] Jul 07 02:52:18.127658 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:18.127412 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" containerID="cri-o://ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17" gracePeriod=30 Jul 07 02:52:19.983115 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:19.983074 2559 patch_prober.go:28] interesting pod/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 02:52:19.983529 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:19.983145 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 02:52:23.385830 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.385774 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:52:23.545441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545405 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-podinfo\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545441 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545440 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9041a6da-c8a4-4871-a381-72ccad53eea2-istiod-ca-cert\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545677 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545524 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-data\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545677 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545545 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-envoy\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545677 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545581 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-token\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545677 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545601 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89nz6\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-kube-api-access-89nz6\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545677 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545636 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-socket\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545677 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545656 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-certs\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545702 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-credential-socket\") pod \"9041a6da-c8a4-4871-a381-72ccad53eea2\" (UID: \"9041a6da-c8a4-4871-a381-72ccad53eea2\") " Jul 07 02:52:23.545987 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.545961 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9041a6da-c8a4-4871-a381-72ccad53eea2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:52:23.546402 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.546212 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:23.546402 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.546250 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:23.546402 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.546361 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:23.546645 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.546547 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-data" (OuterVolumeSpecName: "istio-data") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:23.547905 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.547881 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-kube-api-access-89nz6" (OuterVolumeSpecName: "kube-api-access-89nz6") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "kube-api-access-89nz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:52:23.547905 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.547897 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-token" (OuterVolumeSpecName: "istio-token") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:52:23.548137 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.548113 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:23.548255 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.548185 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9041a6da-c8a4-4871-a381-72ccad53eea2" (UID: "9041a6da-c8a4-4871-a381-72ccad53eea2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646898 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646923 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646932 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646940 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89nz6\" (UniqueName: \"kubernetes.io/projected/9041a6da-c8a4-4871-a381-72ccad53eea2-kube-api-access-89nz6\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646950 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646961 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646969 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9041a6da-c8a4-4871-a381-72ccad53eea2-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646977 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9041a6da-c8a4-4871-a381-72ccad53eea2-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:23.646976 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:23.646984 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9041a6da-c8a4-4871-a381-72ccad53eea2-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:24.168231 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.168192 2559 generic.go:356] "Generic (PLEG): container finished" podID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerID="ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17" exitCode=0 Jul 07 02:52:24.168406 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.168262 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" Jul 07 02:52:24.168406 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.168273 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" event={"ID":"9041a6da-c8a4-4871-a381-72ccad53eea2","Type":"ContainerDied","Data":"ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17"} Jul 07 02:52:24.168406 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.168332 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p" event={"ID":"9041a6da-c8a4-4871-a381-72ccad53eea2","Type":"ContainerDied","Data":"bc9735b09b6ab285979c768e7855e75d3724af31c576cd7daafa0c9f051d1260"} Jul 07 02:52:24.168406 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.168358 2559 scope.go:117] "RemoveContainer" containerID="ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17" Jul 07 02:52:24.178199 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.178181 2559 scope.go:117] "RemoveContainer" containerID="a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e" Jul 07 02:52:24.186831 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.186802 2559 scope.go:117] "RemoveContainer" containerID="ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17" Jul 07 02:52:24.187072 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:52:24.187056 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17\": container with ID starting with ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17 not found: ID does not exist" containerID="ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17" Jul 07 02:52:24.187139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.187079 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17"} err="failed to get container status \"ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17\": rpc error: code = NotFound desc = could not find container \"ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17\": container with ID starting with ebaee38214ef4b47f8106824216170242fbc182a689f025228a6a95698db3a17 not found: ID does not exist" Jul 07 02:52:24.187139 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.187094 2559 scope.go:117] "RemoveContainer" containerID="a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e" Jul 07 02:52:24.187352 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:52:24.187332 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e\": container with ID starting with a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e not found: ID does not exist" containerID="a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e" Jul 07 02:52:24.187393 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.187363 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e"} err="failed to get container status \"a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e\": rpc error: code = NotFound desc = could not find container \"a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e\": container with ID starting with a9c6ec79e2027f20294923cabb399e490e7c687e1fcec50997f8ef73b59a373e not found: ID does not exist" Jul 07 02:52:24.192774 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.192751 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p"] Jul 07 02:52:24.198798 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:24.198778 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f01161-openshift-default-5c4784fdc-msz7p"] Jul 07 02:52:25.206835 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:25.206797 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7"] Jul 07 02:52:25.207260 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:25.207061 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" containerID="cri-o://8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287" gracePeriod=30 Jul 07 02:52:25.818886 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:25.818854 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" path="/var/lib/kubelet/pods/9041a6da-c8a4-4871-a381-72ccad53eea2/volumes" Jul 07 02:52:25.907525 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:25.907496 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dbbb754f5-7lxs7_6836ceaa-e27d-4b28-badf-77714eaf9c39/authorino/0.log" Jul 07 02:52:27.345883 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:27.345843 2559 patch_prober.go:28] interesting pod/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 02:52:27.346286 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:27.345908 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 02:52:30.464521 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.464495 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:52:30.609547 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609464 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-podinfo\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.609547 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609509 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-socket\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.609781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609566 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-envoy\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.609781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609608 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-certs\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.609781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609634 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkl2g\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-kube-api-access-fkl2g\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.609781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609665 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istiod-ca-cert\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.609781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609683 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-credential-socket\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.609781 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609698 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-token\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.610107 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609871 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:30.610107 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.609998 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:30.610107 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610008 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:30.610107 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610074 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-data\") pod \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\" (UID: \"6b304a3a-b41d-4bb7-b337-3a48be717aa1\") " Jul 07 02:52:30.610107 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610097 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 02:52:30.610483 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610462 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istiod-ca-cert\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.610550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610486 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-credential-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.610550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610499 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-socket\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.610550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610511 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-workload-certs\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.610823 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.610804 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:30.611855 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.611827 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 02:52:30.611970 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.611888 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-kube-api-access-fkl2g" (OuterVolumeSpecName: "kube-api-access-fkl2g") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "kube-api-access-fkl2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:52:30.612048 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.612031 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 02:52:30.612182 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.612165 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b304a3a-b41d-4bb7-b337-3a48be717aa1" (UID: "6b304a3a-b41d-4bb7-b337-3a48be717aa1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 02:52:30.711765 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.711730 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-token\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.711765 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.711758 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-data\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.711765 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.711766 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-podinfo\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.711765 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.711775 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b304a3a-b41d-4bb7-b337-3a48be717aa1-istio-envoy\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.712032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.711786 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkl2g\" (UniqueName: \"kubernetes.io/projected/6b304a3a-b41d-4bb7-b337-3a48be717aa1-kube-api-access-fkl2g\") on node \"ip-10-0-141-30.ec2.internal\" DevicePath \"\"" Jul 07 02:52:30.826679 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:30.826650 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-cc8d47df9-lmg2v_4309c267-cdfe-4259-8b42-86c8779dabde/maas-api/0.log" Jul 07 02:52:31.211491 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.211457 2559 generic.go:356] "Generic (PLEG): container finished" podID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerID="8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287" exitCode=0 Jul 07 02:52:31.211689 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.211508 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" Jul 07 02:52:31.211689 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.211508 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" event={"ID":"6b304a3a-b41d-4bb7-b337-3a48be717aa1","Type":"ContainerDied","Data":"8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287"} Jul 07 02:52:31.211689 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.211649 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7" event={"ID":"6b304a3a-b41d-4bb7-b337-3a48be717aa1","Type":"ContainerDied","Data":"6203dd254176d75b664fddae98253886b6fbf3d0233e27ca9547292b2b3f46f2"} Jul 07 02:52:31.211689 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.211683 2559 scope.go:117] "RemoveContainer" containerID="8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287" Jul 07 02:52:31.221971 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.221951 2559 scope.go:117] "RemoveContainer" containerID="9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff" Jul 07 02:52:31.230200 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.230182 2559 scope.go:117] "RemoveContainer" containerID="8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287" Jul 07 02:52:31.230431 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:52:31.230411 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287\": container with ID starting with 8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287 not found: ID does not exist" containerID="8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287" Jul 07 02:52:31.230478 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.230439 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287"} err="failed to get container status \"8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287\": rpc error: code = NotFound desc = could not find container \"8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287\": container with ID starting with 8f9b6bd86feeddf8cb352853feaaea6ca22a37914cc464c02e2c7a6a2e44e287 not found: ID does not exist" Jul 07 02:52:31.230478 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.230456 2559 scope.go:117] "RemoveContainer" containerID="9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff" Jul 07 02:52:31.230718 ip-10-0-141-30 kubenswrapper[2559]: E0707 02:52:31.230700 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff\": container with ID starting with 9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff not found: ID does not exist" containerID="9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff" Jul 07 02:52:31.230769 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.230723 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff"} err="failed to get container status \"9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff\": rpc error: code = NotFound desc = could not find container \"9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff\": container with ID starting with 9dff478f870c1f6634090fa3813161017caa2673ced474c0ab8bd3f40f2c8dff not found: ID does not exist" Jul 07 02:52:31.234701 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.234679 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7"] Jul 07 02:52:31.237996 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.237978 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3dabf0-openshift-default-6cddfffb87-r9vq7"] Jul 07 02:52:31.662329 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.662302 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-k9tkh_6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52/copy-manifests/0.log" Jul 07 02:52:31.674803 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.674781 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-k9tkh_6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52/manager/0.log" Jul 07 02:52:31.819012 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.818979 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" path="/var/lib/kubelet/pods/6b304a3a-b41d-4bb7-b337-3a48be717aa1/volumes" Jul 07 02:52:31.910733 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:31.910702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-vznhk_eead3123-e443-4af4-afbb-e0f4d6455ecb/postgres/0.log" Jul 07 02:52:32.710937 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.710905 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq_61bc2748-4d91-4d73-a339-e60011fe4670/util/0.log" Jul 07 02:52:32.717308 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.717288 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq_61bc2748-4d91-4d73-a339-e60011fe4670/pull/0.log" Jul 07 02:52:32.723401 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.723383 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq_61bc2748-4d91-4d73-a339-e60011fe4670/extract/0.log" Jul 07 02:52:32.839982 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.839953 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2_90671ba1-dec9-42e4-8c3f-d24a253b17cc/util/0.log" Jul 07 02:52:32.846658 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.846641 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2_90671ba1-dec9-42e4-8c3f-d24a253b17cc/pull/0.log" Jul 07 02:52:32.853561 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.853543 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2_90671ba1-dec9-42e4-8c3f-d24a253b17cc/extract/0.log" Jul 07 02:52:32.970195 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.970102 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn_8461e185-e859-40de-93a8-a136086d7639/pull/0.log" Jul 07 02:52:32.976251 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.976233 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn_8461e185-e859-40de-93a8-a136086d7639/extract/0.log" Jul 07 02:52:32.982187 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:32.982172 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn_8461e185-e859-40de-93a8-a136086d7639/util/0.log" Jul 07 02:52:33.099033 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:33.099005 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw_49d0c21e-e32f-41fc-b548-6143959ee966/pull/0.log" Jul 07 02:52:33.105591 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:33.105576 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw_49d0c21e-e32f-41fc-b548-6143959ee966/extract/0.log" Jul 07 02:52:33.112031 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:33.112015 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw_49d0c21e-e32f-41fc-b548-6143959ee966/util/0.log" Jul 07 02:52:33.229434 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:33.229353 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dbbb754f5-7lxs7_6836ceaa-e27d-4b28-badf-77714eaf9c39/authorino/0.log" Jul 07 02:52:33.359337 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:33.359308 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wrrdl_9a87265c-48a1-45a4-a73b-236653a87e37/manager/0.log" Jul 07 02:52:33.590789 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:33.590716 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z4nd2_419c705b-aaa6-4472-a85a-57d2ce84f40f/kuadrant-console-plugin/0.log" Jul 07 02:52:33.715852 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:33.715822 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ptn4f_3a5507c1-54ad-41cf-9a10-4667987fdf4d/registry-server/0.log" Jul 07 02:52:34.453867 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:34.453841 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45wlpl_f846a0a8-35c2-461f-9fe1-dd85bed1c26a/istio-proxy/0.log" Jul 07 02:52:34.935322 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:34.935293 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hvnvp_f2e7d479-6c21-4872-b695-ac491c96d1e6/istio-proxy/1.log" Jul 07 02:52:35.053751 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:35.053718 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-p52nl_e1caa62c-7e05-4161-a0e8-a6e619f7c16f/payload-pre-processing/0.log" Jul 07 02:52:35.171141 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:35.171110 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zslkb_c358ad22-3d77-4895-b7af-f7e230131dac/payload-processing/0.log" Jul 07 02:52:35.288108 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:35.288033 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78d788fd9-7ww7b_5bea1683-234d-447f-87b4-fbdf9186358b/router/0.log" Jul 07 02:52:35.889550 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:35.889513 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-s5mbz_c66f979a-97c8-4a34-b3c6-576605cb1a08/main/0.log" Jul 07 02:52:36.113419 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:36.113380 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-px787_97743ae2-f73a-4494-8ccf-356f9281b0b5/main/0.log" Jul 07 02:52:42.909826 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:42.909797 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d75g5_b459b352-3f73-4a90-a434-440bcb843220/global-pull-secret-syncer/0.log" Jul 07 02:52:43.057907 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:43.057875 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wvvwk_535498bf-bd89-46dd-9937-2f3adce9941b/konnectivity-agent/0.log" Jul 07 02:52:43.104287 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:43.104262 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-30.ec2.internal_419e48f62642a0af69b25fe0fa0d0553/haproxy/0.log" Jul 07 02:52:47.139012 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.138939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq_61bc2748-4d91-4d73-a339-e60011fe4670/extract/0.log" Jul 07 02:52:47.161519 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.161491 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq_61bc2748-4d91-4d73-a339-e60011fe4670/util/0.log" Jul 07 02:52:47.184710 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.184677 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nm7lq_61bc2748-4d91-4d73-a339-e60011fe4670/pull/0.log" Jul 07 02:52:47.212975 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.212937 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2_90671ba1-dec9-42e4-8c3f-d24a253b17cc/extract/0.log" Jul 07 02:52:47.239021 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.239002 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2_90671ba1-dec9-42e4-8c3f-d24a253b17cc/util/0.log" Jul 07 02:52:47.262102 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.262074 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0q8nd2_90671ba1-dec9-42e4-8c3f-d24a253b17cc/pull/0.log" Jul 07 02:52:47.289740 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.289719 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn_8461e185-e859-40de-93a8-a136086d7639/extract/0.log" Jul 07 02:52:47.322533 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.322510 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn_8461e185-e859-40de-93a8-a136086d7639/util/0.log" Jul 07 02:52:47.350188 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.350165 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73br2vn_8461e185-e859-40de-93a8-a136086d7639/pull/0.log" Jul 07 02:52:47.378579 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.378559 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw_49d0c21e-e32f-41fc-b548-6143959ee966/extract/0.log" Jul 07 02:52:47.400385 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.400360 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw_49d0c21e-e32f-41fc-b548-6143959ee966/util/0.log" Jul 07 02:52:47.432939 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.432913 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t5nqw_49d0c21e-e32f-41fc-b548-6143959ee966/pull/0.log" Jul 07 02:52:47.702500 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.702411 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dbbb754f5-7lxs7_6836ceaa-e27d-4b28-badf-77714eaf9c39/authorino/0.log" Jul 07 02:52:47.742623 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.742597 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wrrdl_9a87265c-48a1-45a4-a73b-236653a87e37/manager/0.log" Jul 07 02:52:47.810119 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.810092 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-z4nd2_419c705b-aaa6-4472-a85a-57d2ce84f40f/kuadrant-console-plugin/0.log" Jul 07 02:52:47.853933 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:47.853904 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ptn4f_3a5507c1-54ad-41cf-9a10-4667987fdf4d/registry-server/0.log" Jul 07 02:52:49.904130 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:49.904099 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-z8797_b54d838d-740f-4e40-9996-08f122a41670/cluster-monitoring-operator/0.log" Jul 07 02:52:49.941246 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:49.941221 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-clz54_cb51ad16-a72d-49bc-a870-90a38a85c58e/kube-state-metrics/0.log" Jul 07 02:52:49.973747 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:49.973721 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-clz54_cb51ad16-a72d-49bc-a870-90a38a85c58e/kube-rbac-proxy-main/0.log" Jul 07 02:52:50.012662 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.012638 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-clz54_cb51ad16-a72d-49bc-a870-90a38a85c58e/kube-rbac-proxy-self/0.log" Jul 07 02:52:50.312841 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.312808 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjcl9_5cb0b590-6421-4d6d-8594-678c9c5d8d9a/node-exporter/0.log" Jul 07 02:52:50.348809 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.348783 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjcl9_5cb0b590-6421-4d6d-8594-678c9c5d8d9a/kube-rbac-proxy/0.log" Jul 07 02:52:50.392816 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.392792 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjcl9_5cb0b590-6421-4d6d-8594-678c9c5d8d9a/init-textfile/0.log" Jul 07 02:52:50.552014 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.551922 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fwsvf_05f26392-b740-44ae-9b89-2b619bf837c0/kube-rbac-proxy-main/0.log" Jul 07 02:52:50.591032 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.591002 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fwsvf_05f26392-b740-44ae-9b89-2b619bf837c0/kube-rbac-proxy-self/0.log" Jul 07 02:52:50.626990 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.626960 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fwsvf_05f26392-b740-44ae-9b89-2b619bf837c0/openshift-state-metrics/0.log" Jul 07 02:52:50.887370 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.887294 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-p6zgv_fa2668d1-3349-41b4-88e8-f5da2216074c/prometheus-operator/0.log" Jul 07 02:52:50.909291 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.909261 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-p6zgv_fa2668d1-3349-41b4-88e8-f5da2216074c/kube-rbac-proxy/0.log" Jul 07 02:52:50.934848 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.934817 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-wgr4c_7155e4ed-cdb5-4b92-889f-d5fe40fe0f18/prometheus-operator-admission-webhook/0.log" Jul 07 02:52:50.972602 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.972574 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f9dc9ffc-spl24_1ae94cd8-aecf-4414-86c5-27730557185a/telemeter-client/0.log" Jul 07 02:52:50.994849 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:50.994815 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f9dc9ffc-spl24_1ae94cd8-aecf-4414-86c5-27730557185a/reload/0.log" Jul 07 02:52:51.022002 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:51.021976 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f9dc9ffc-spl24_1ae94cd8-aecf-4414-86c5-27730557185a/kube-rbac-proxy/0.log" Jul 07 02:52:53.713292 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713250 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk"] Jul 07 02:52:53.713807 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713788 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" Jul 07 02:52:53.713893 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713811 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" Jul 07 02:52:53.713893 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713822 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" Jul 07 02:52:53.713893 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713831 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" Jul 07 02:52:53.713893 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713868 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" Jul 07 02:52:53.713893 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713877 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" Jul 07 02:52:53.714146 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713897 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" Jul 07 02:52:53.714146 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.713906 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" Jul 07 02:52:53.714146 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.714004 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b304a3a-b41d-4bb7-b337-3a48be717aa1" containerName="istio-proxy" Jul 07 02:52:53.714146 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.714017 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" Jul 07 02:52:53.714146 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.714027 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fcdd2ac-082c-4653-be56-348f9822242e" containerName="istio-proxy" Jul 07 02:52:53.714146 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.714037 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9041a6da-c8a4-4871-a381-72ccad53eea2" containerName="istio-proxy" Jul 07 02:52:53.716408 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.716387 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.719683 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.719663 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7l5m7\"/\"openshift-service-ca.crt\"" Jul 07 02:52:53.720389 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.720370 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7l5m7\"/\"default-dockercfg-rbjgn\"" Jul 07 02:52:53.720389 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.720382 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7l5m7\"/\"kube-root-ca.crt\"" Jul 07 02:52:53.724721 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.724696 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk"] Jul 07 02:52:53.828100 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.828073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnrdf\" (UniqueName: \"kubernetes.io/projected/849c21d9-abc7-47e6-9802-963f7c570705-kube-api-access-lnrdf\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.828271 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.828106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-sys\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.828271 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.828143 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-proc\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.828271 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.828246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-podres\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.828391 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.828356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-lib-modules\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929282 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929248 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrdf\" (UniqueName: \"kubernetes.io/projected/849c21d9-abc7-47e6-9802-963f7c570705-kube-api-access-lnrdf\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929291 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-sys\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929326 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-proc\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929360 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-podres\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-sys\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929469 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929417 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-lib-modules\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929467 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-proc\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929538 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-lib-modules\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.929646 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.929538 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/849c21d9-abc7-47e6-9802-963f7c570705-podres\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:53.937333 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:53.937306 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnrdf\" (UniqueName: \"kubernetes.io/projected/849c21d9-abc7-47e6-9802-963f7c570705-kube-api-access-lnrdf\") pod \"perf-node-gather-daemonset-kfwfk\" (UID: \"849c21d9-abc7-47e6-9802-963f7c570705\") " pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:54.028321 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:54.028235 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:54.158243 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:54.158219 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk"] Jul 07 02:52:54.159825 ip-10-0-141-30 kubenswrapper[2559]: W0707 02:52:54.159795 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod849c21d9_abc7_47e6_9802_963f7c570705.slice/crio-21f9835e94646d296f1150deeea16999e3c39af54d2cc6f58bdf5db7cef194e5 WatchSource:0}: Error finding container 21f9835e94646d296f1150deeea16999e3c39af54d2cc6f58bdf5db7cef194e5: Status 404 returned error can't find the container with id 21f9835e94646d296f1150deeea16999e3c39af54d2cc6f58bdf5db7cef194e5 Jul 07 02:52:54.303899 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:54.303823 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-6xzd4_a7ece919-26cf-403e-90a9-bd15239a5e87/volume-data-source-validator/0.log" Jul 07 02:52:54.334627 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:54.334599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" event={"ID":"849c21d9-abc7-47e6-9802-963f7c570705","Type":"ContainerStarted","Data":"53363cfd85d641d13d019a2346c9fcb2f34a35fcb34d3925c6ac18902097d16a"} Jul 07 02:52:54.334755 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:54.334652 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:52:54.334755 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:54.334676 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" event={"ID":"849c21d9-abc7-47e6-9802-963f7c570705","Type":"ContainerStarted","Data":"21f9835e94646d296f1150deeea16999e3c39af54d2cc6f58bdf5db7cef194e5"} Jul 07 02:52:54.350856 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:54.350808 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" podStartSLOduration=1.3507914429999999 podStartE2EDuration="1.350791443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 02:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 02:52:54.349827629 +0000 UTC m=+2591.100249283" watchObservedRunningTime="2026-07-07 02:52:54.350791443 +0000 UTC m=+2591.101213096" Jul 07 02:52:55.342329 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:55.342293 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2276z_b685c185-7f9e-4e17-ac8c-1906d1cd44ae/dns/0.log" Jul 07 02:52:55.363952 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:55.363928 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2276z_b685c185-7f9e-4e17-ac8c-1906d1cd44ae/kube-rbac-proxy/0.log" Jul 07 02:52:55.476925 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:55.476896 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dcf4t_6dd64963-deab-4e30-973a-d6cebc6e0209/dns-node-resolver/0.log" Jul 07 02:52:56.053380 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:56.053342 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-746df6866f-zcg6t_17780b36-bc96-4268-99e5-0b71b432a3f4/registry/0.log" Jul 07 02:52:56.074898 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:56.074857 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b4gg5_6684a217-290d-489a-8514-9b48a7940479/node-ca/0.log" Jul 07 02:52:57.037226 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:57.037192 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45wlpl_f846a0a8-35c2-461f-9fe1-dd85bed1c26a/istio-proxy/0.log" Jul 07 02:52:57.227171 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:57.227134 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hvnvp_f2e7d479-6c21-4872-b695-ac491c96d1e6/istio-proxy/0.log" Jul 07 02:52:57.253559 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:57.253529 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hvnvp_f2e7d479-6c21-4872-b695-ac491c96d1e6/istio-proxy/1.log" Jul 07 02:52:57.285406 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:57.285381 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-p52nl_e1caa62c-7e05-4161-a0e8-a6e619f7c16f/payload-pre-processing/0.log" Jul 07 02:52:57.335277 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:57.335198 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zslkb_c358ad22-3d77-4895-b7af-f7e230131dac/payload-processing/0.log" Jul 07 02:52:57.357307 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:57.357282 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78d788fd9-7ww7b_5bea1683-234d-447f-87b4-fbdf9186358b/router/0.log" Jul 07 02:52:58.005988 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:58.005961 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6fc7h_e37a17b2-5148-48ab-aa5b-36dcd2569c84/serve-healthcheck-canary/0.log" Jul 07 02:52:58.553618 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:58.553594 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-htqsl_2ad31900-bfd3-4b69-9598-7fdfbb902fa7/kube-rbac-proxy/0.log" Jul 07 02:52:58.576411 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:58.576390 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-htqsl_2ad31900-bfd3-4b69-9598-7fdfbb902fa7/exporter/0.log" Jul 07 02:52:58.597775 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:52:58.597749 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-htqsl_2ad31900-bfd3-4b69-9598-7fdfbb902fa7/extractor/0.log" Jul 07 02:53:00.350801 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:00.350773 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7l5m7/perf-node-gather-daemonset-kfwfk" Jul 07 02:53:01.145614 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:01.145583 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-cc8d47df9-lmg2v_4309c267-cdfe-4259-8b42-86c8779dabde/maas-api/0.log" Jul 07 02:53:01.493822 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:01.493785 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-k9tkh_6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52/manager/0.log" Jul 07 02:53:01.511624 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:01.511597 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76797ccd4c-k9tkh_6bf320b7-9a36-4cfa-b1c6-d04e0c7e8c52/copy-manifests/0.log" Jul 07 02:53:01.584728 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:01.584702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-vznhk_eead3123-e443-4af4-afbb-e0f4d6455ecb/postgres/0.log" Jul 07 02:53:02.805558 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:02.805530 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bd684c58c-7l2j7_8b58fbd5-e83e-4278-9d7d-8e054d66066c/manager/0.log" Jul 07 02:53:03.477658 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:03.477626 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45wlpl_f846a0a8-35c2-461f-9fe1-dd85bed1c26a/istio-proxy/0.log" Jul 07 02:53:03.648722 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:03.648697 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hvnvp_f2e7d479-6c21-4872-b695-ac491c96d1e6/istio-proxy/0.log" Jul 07 02:53:03.674588 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:03.674559 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hvnvp_f2e7d479-6c21-4872-b695-ac491c96d1e6/istio-proxy/1.log" Jul 07 02:53:03.698735 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:03.698708 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-p52nl_e1caa62c-7e05-4161-a0e8-a6e619f7c16f/payload-pre-processing/0.log" Jul 07 02:53:03.735834 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:03.735761 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zslkb_c358ad22-3d77-4895-b7af-f7e230131dac/payload-processing/0.log" Jul 07 02:53:03.758524 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:03.758496 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78d788fd9-7ww7b_5bea1683-234d-447f-87b4-fbdf9186358b/router/0.log" Jul 07 02:53:05.157489 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:05.157465 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45wlpl_f846a0a8-35c2-461f-9fe1-dd85bed1c26a/istio-proxy/0.log" Jul 07 02:53:05.310913 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:05.310883 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hvnvp_f2e7d479-6c21-4872-b695-ac491c96d1e6/istio-proxy/0.log" Jul 07 02:53:05.336887 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:05.336860 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hvnvp_f2e7d479-6c21-4872-b695-ac491c96d1e6/istio-proxy/1.log" Jul 07 02:53:05.359572 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:05.359548 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-p52nl_e1caa62c-7e05-4161-a0e8-a6e619f7c16f/payload-pre-processing/0.log" Jul 07 02:53:05.399138 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:05.399115 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zslkb_c358ad22-3d77-4895-b7af-f7e230131dac/payload-processing/0.log" Jul 07 02:53:05.414727 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:05.414655 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78d788fd9-7ww7b_5bea1683-234d-447f-87b4-fbdf9186358b/router/0.log" Jul 07 02:53:08.225185 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:08.225139 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kscgp_6325b1d9-f4d3-480d-bdb1-e379b30d64ae/migrator/0.log" Jul 07 02:53:08.247024 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:08.247001 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kscgp_6325b1d9-f4d3-480d-bdb1-e379b30d64ae/graceful-termination/0.log" Jul 07 02:53:08.612551 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:08.612478 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-chj6c_a3df569c-4759-44bc-82fa-143c559207a8/kube-storage-version-migrator-operator/1.log" Jul 07 02:53:08.613268 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:08.613250 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-chj6c_a3df569c-4759-44bc-82fa-143c559207a8/kube-storage-version-migrator-operator/0.log" Jul 07 02:53:09.664870 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:09.664844 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hhh5n_b0278ab3-1cd5-4503-8554-43f8bc4c8398/kube-multus-additional-cni-plugins/0.log" Jul 07 02:53:09.687937 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:09.687910 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hhh5n_b0278ab3-1cd5-4503-8554-43f8bc4c8398/egress-router-binary-copy/0.log" Jul 07 02:53:09.709203 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:09.709147 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hhh5n_b0278ab3-1cd5-4503-8554-43f8bc4c8398/cni-plugins/0.log" Jul 07 02:53:09.728477 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:09.728461 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hhh5n_b0278ab3-1cd5-4503-8554-43f8bc4c8398/bond-cni-plugin/0.log" Jul 07 02:53:09.748462 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:09.748438 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hhh5n_b0278ab3-1cd5-4503-8554-43f8bc4c8398/routeoverride-cni/0.log" Jul 07 02:53:09.770548 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:09.770528 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hhh5n_b0278ab3-1cd5-4503-8554-43f8bc4c8398/whereabouts-cni-bincopy/0.log" Jul 07 02:53:09.795827 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:09.795803 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hhh5n_b0278ab3-1cd5-4503-8554-43f8bc4c8398/whereabouts-cni/0.log" Jul 07 02:53:10.155382 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:10.155359 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dzhkk_a395fe49-f597-4ebe-bf24-38af015944f6/kube-multus/0.log" Jul 07 02:53:10.212950 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:10.212924 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44l58_af9fae31-f4c3-4823-b486-19c0322111bb/network-metrics-daemon/0.log" Jul 07 02:53:10.232741 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:10.232702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-44l58_af9fae31-f4c3-4823-b486-19c0322111bb/kube-rbac-proxy/0.log" Jul 07 02:53:11.376763 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.376685 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/ovn-controller/0.log" Jul 07 02:53:11.394974 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.394949 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/ovn-acl-logging/0.log" Jul 07 02:53:11.424238 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.424212 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/kube-rbac-proxy-node/0.log" Jul 07 02:53:11.446886 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.446858 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/kube-rbac-proxy-ovn-metrics/0.log" Jul 07 02:53:11.472376 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.472347 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/northd/0.log" Jul 07 02:53:11.495062 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.495037 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/nbdb/0.log" Jul 07 02:53:11.517708 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.517686 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/sbdb/0.log" Jul 07 02:53:11.568839 ip-10-0-141-30 kubenswrapper[2559]: I0707 02:53:11.568806 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9c6j6_b59a8c49-adcb-4435-8667-41a2203d706c/ovnkube-controller/0.log"