Jul 23 15:48:05.411808 ip-10-0-139-161 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 23 15:48:05.411816 ip-10-0-139-161 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 23 15:48:05.411823 ip-10-0-139-161 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 23 15:48:05.412060 ip-10-0-139-161 systemd[1]: Failed to start Kubernetes Kubelet. Jul 23 15:48:15.606234 ip-10-0-139-161 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 23 15:48:15.606253 ip-10-0-139-161 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 70bad63c795346f3b9ab9ad4a7fe5d12 -- Jul 23 15:50:12.131687 ip-10-0-139-161 systemd[1]: Starting Kubernetes Kubelet... Jul 23 15:50:14.007809 ip-10-0-139-161 kubenswrapper[2547]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 15:50:14.007809 ip-10-0-139-161 kubenswrapper[2547]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 15:50:14.007809 ip-10-0-139-161 kubenswrapper[2547]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 15:50:14.007809 ip-10-0-139-161 kubenswrapper[2547]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 15:50:14.008307 ip-10-0-139-161 kubenswrapper[2547]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 15:50:14.008808 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.008734 2547 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 15:50:14.014175 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014154 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:50:14.014175 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014170 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:50:14.014175 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014174 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:50:14.014175 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014177 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:50:14.014175 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014180 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014183 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014186 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014189 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014193 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014196 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014198 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014201 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014204 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014206 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014209 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014211 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014214 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014216 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014219 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014222 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014225 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014227 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014230 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014234 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:50:14.014345 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014238 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014242 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014246 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014249 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014251 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014254 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014257 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014260 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014262 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014265 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014268 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014270 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014273 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014276 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014279 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014282 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014285 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014288 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014290 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014293 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:50:14.014835 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014295 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014298 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014301 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014303 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014307 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014309 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014313 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014315 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014319 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014323 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014326 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014329 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014332 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014335 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014338 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014340 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014343 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014346 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014349 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:50:14.015342 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014352 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014355 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014358 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014360 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014363 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014366 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014369 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014372 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014375 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014378 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014380 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014383 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014385 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014388 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014390 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014393 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014396 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014398 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014401 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014403 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:50:14.015814 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014406 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014408 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014412 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014414 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014417 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014419 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014422 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014425 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014427 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014430 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014432 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014435 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014438 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014442 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014445 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:50:14.016286 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.014447 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016498 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016505 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016509 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016512 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016515 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016518 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016521 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016524 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016527 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016529 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016532 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016534 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016537 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016539 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016542 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016544 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016547 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016550 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016553 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:50:14.016671 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016555 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016557 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016560 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016562 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016565 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016567 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016570 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016572 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016574 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016577 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016580 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016582 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016584 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016587 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016591 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016595 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016599 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016602 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016605 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:50:14.017166 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016608 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016612 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016616 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016619 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016622 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016624 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016627 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016629 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016632 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016634 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016637 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016639 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016642 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016645 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016647 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016650 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016652 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016655 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016657 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016660 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:50:14.017646 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016662 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016665 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016667 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016670 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016673 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016677 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016680 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016683 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016685 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016705 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016708 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016711 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016714 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016716 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016719 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016722 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016724 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016727 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016731 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016734 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:50:14.018138 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016736 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016739 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016741 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016743 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016746 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016749 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016751 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016755 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016759 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016762 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016765 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016767 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016770 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016773 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016775 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016778 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016781 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016783 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016786 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:50:14.018615 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016789 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.016792 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017164 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017169 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017172 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017175 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017178 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017180 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017183 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017185 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017188 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017190 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017193 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017196 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017198 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017201 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017203 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017206 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017209 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017212 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:50:14.019092 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017214 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017217 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017220 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017223 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017225 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017227 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017231 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017235 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017238 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017241 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017243 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017246 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017249 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017252 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017254 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017257 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017259 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017262 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017264 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017267 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:50:14.019561 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017269 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017272 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017274 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017277 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017281 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017286 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017289 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017292 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017295 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017298 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017300 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017304 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017306 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017309 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017312 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017314 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017317 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017319 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017322 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:50:14.020065 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017324 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017327 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017331 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017334 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017337 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017339 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017342 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017344 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017347 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017349 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017351 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017354 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017356 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017359 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017361 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017364 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017366 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017370 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017374 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:50:14.020514 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017377 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017380 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017382 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017385 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017387 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017390 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017393 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017395 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017398 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017400 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017403 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017405 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017407 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017410 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017412 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017415 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017418 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017420 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017423 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017426 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:50:14.020981 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017429 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017431 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.017434 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017504 2547 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017511 2547 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017516 2547 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017520 2547 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017524 2547 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017528 2547 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017532 2547 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017536 2547 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017540 2547 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017542 2547 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017546 2547 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017549 2547 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017552 2547 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017555 2547 flags.go:64] FLAG: --cgroup-root="" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017558 2547 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017561 2547 flags.go:64] FLAG: --client-ca-file="" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017564 2547 flags.go:64] FLAG: --cloud-provider="external" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017567 2547 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017591 2547 flags.go:64] FLAG: --cluster-domain="" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017595 2547 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017599 2547 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 15:50:14.021436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017602 2547 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017605 2547 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017609 2547 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017613 2547 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017616 2547 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017620 2547 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017623 2547 flags.go:64] FLAG: --contention-profiling="false" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017626 2547 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017629 2547 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017632 2547 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017635 2547 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017638 2547 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017641 2547 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017644 2547 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017647 2547 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017650 2547 flags.go:64] FLAG: --enable-server="true" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017653 2547 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017656 2547 flags.go:64] FLAG: --event-burst="100" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017660 2547 flags.go:64] FLAG: --event-qps="50" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017663 2547 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017666 2547 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017668 2547 flags.go:64] FLAG: --eviction-hard="" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017672 2547 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017675 2547 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017678 2547 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 15:50:14.022021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017681 2547 flags.go:64] FLAG: --eviction-soft="" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017683 2547 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017686 2547 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017706 2547 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017711 2547 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017714 2547 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017717 2547 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017720 2547 flags.go:64] FLAG: --feature-gates="" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017724 2547 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017727 2547 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017731 2547 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017734 2547 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017737 2547 flags.go:64] FLAG: --healthz-port="10248" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017740 2547 flags.go:64] FLAG: --help="false" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017743 2547 flags.go:64] FLAG: --hostname-override="ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017746 2547 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017749 2547 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017752 2547 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017755 2547 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017758 2547 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017761 2547 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017764 2547 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017767 2547 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 15:50:14.022601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017769 2547 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017772 2547 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017775 2547 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017778 2547 flags.go:64] FLAG: --kube-reserved="" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017781 2547 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017784 2547 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017787 2547 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017789 2547 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017792 2547 flags.go:64] FLAG: --lock-file="" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017795 2547 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017798 2547 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017801 2547 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017806 2547 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017810 2547 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017813 2547 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017816 2547 flags.go:64] FLAG: --logging-format="text" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017819 2547 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017822 2547 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017825 2547 flags.go:64] FLAG: --manifest-url="" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017829 2547 flags.go:64] FLAG: --manifest-url-header="" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017833 2547 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017836 2547 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017840 2547 flags.go:64] FLAG: --max-pods="110" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017843 2547 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017846 2547 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 15:50:14.023218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017849 2547 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017851 2547 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017854 2547 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017857 2547 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017860 2547 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017868 2547 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017871 2547 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017873 2547 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017876 2547 flags.go:64] FLAG: --pod-cidr="" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017879 2547 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017885 2547 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017888 2547 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017891 2547 flags.go:64] FLAG: --pods-per-core="0" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017894 2547 flags.go:64] FLAG: --port="10250" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017897 2547 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017900 2547 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-038f45750bb1885db" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017903 2547 flags.go:64] FLAG: --qos-reserved="" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017906 2547 flags.go:64] FLAG: --read-only-port="10255" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017909 2547 flags.go:64] FLAG: --register-node="true" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017911 2547 flags.go:64] FLAG: --register-with-taints="" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017915 2547 flags.go:64] FLAG: --registry-burst="10" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017920 2547 flags.go:64] FLAG: --registry-qps="5" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017923 2547 flags.go:64] FLAG: --reserved-cpus="" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017926 2547 flags.go:64] FLAG: --reserved-memory="" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017929 2547 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 15:50:14.023836 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017932 2547 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017937 2547 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017940 2547 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017943 2547 flags.go:64] FLAG: --runonce="false" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017945 2547 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017948 2547 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017951 2547 flags.go:64] FLAG: --seccomp-default="false" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017954 2547 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017957 2547 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017960 2547 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017963 2547 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017966 2547 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017969 2547 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017971 2547 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017974 2547 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017977 2547 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017980 2547 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017983 2547 flags.go:64] FLAG: --system-cgroups="" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017985 2547 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017990 2547 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017993 2547 flags.go:64] FLAG: --tls-cert-file="" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.017996 2547 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018000 2547 flags.go:64] FLAG: --tls-min-version="" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018002 2547 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018005 2547 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 15:50:14.024491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018008 2547 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018011 2547 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018014 2547 flags.go:64] FLAG: --v="2" Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018019 2547 flags.go:64] FLAG: --version="false" Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018026 2547 flags.go:64] FLAG: --vmodule="" Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018030 2547 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.018033 2547 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018139 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018145 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018148 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018152 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018154 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018159 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018162 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018165 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018169 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018172 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018174 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018178 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018181 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018183 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:50:14.025109 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018186 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018189 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018192 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018195 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018198 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018201 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018204 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018207 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018209 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018212 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018214 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018217 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018219 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018222 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018226 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018229 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018232 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018234 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018237 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018240 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:50:14.025612 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018243 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018245 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018248 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018251 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018253 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018255 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018258 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018261 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018263 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018266 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018268 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018270 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018273 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018276 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018278 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018280 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018283 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018285 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018288 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018291 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:50:14.026097 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018293 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018295 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018298 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018300 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018303 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018305 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018309 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018312 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018314 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018317 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018320 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018323 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018325 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018328 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018331 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018333 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018337 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018341 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:50:14.026622 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018345 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018348 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018351 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018354 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018356 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018359 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018362 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018364 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018367 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018369 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018372 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018374 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018377 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018379 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018382 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018385 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018387 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018390 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018392 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018395 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:50:14.027114 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018398 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018401 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018403 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018406 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018409 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018412 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.018414 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.019267 2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.025467 2547 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.025482 2547 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.025495 2547 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.025504 2547 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025559 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025564 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025568 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:50:14.027595 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025572 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025575 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025577 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025580 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025582 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025585 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025588 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025591 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025594 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025597 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025600 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025602 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025605 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025608 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025610 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025613 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025615 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025618 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025621 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025623 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:50:14.027978 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025626 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025629 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025631 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025634 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025636 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025639 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025643 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025645 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025648 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025651 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025653 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025656 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025658 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025661 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025663 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025666 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025668 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025671 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025675 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025678 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:50:14.028477 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025681 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025684 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025686 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025703 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025706 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025709 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025711 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025714 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025717 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025719 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025722 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025724 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025727 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025730 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025732 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025735 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025737 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025741 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025745 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:50:14.028976 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025749 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025752 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025755 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025759 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025761 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025764 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025767 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025770 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025773 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025775 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025778 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025781 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025784 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025787 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025791 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025793 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025796 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025799 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025801 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:50:14.029437 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025804 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025806 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025809 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025812 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025814 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025817 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025819 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025821 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025824 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025826 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025829 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025832 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025836 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025839 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025843 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025845 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025848 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:50:14.030086 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025851 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.025856 2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025985 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025990 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025994 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.025997 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026000 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026003 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026006 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026009 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026011 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026014 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026016 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026019 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026021 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 15:50:14.030496 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026024 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026026 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026029 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026031 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026034 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026036 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026039 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026041 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026044 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026046 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026049 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026052 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026055 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026057 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026060 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026063 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026065 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026068 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026071 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026074 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 15:50:14.030958 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026076 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026079 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026083 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026087 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026089 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026092 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026094 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026097 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026100 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026102 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026105 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026108 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026110 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026113 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026116 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026118 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026121 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026123 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026126 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 15:50:14.031455 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026128 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026131 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026133 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026136 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026138 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026141 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026143 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026146 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026148 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026153 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026156 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026159 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026162 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026165 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026167 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026170 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026173 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026175 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026178 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 15:50:14.032001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026180 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026183 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026185 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026188 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026190 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026193 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026195 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026197 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026200 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026203 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026205 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026208 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026210 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026213 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026215 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026217 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026220 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026223 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026225 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026228 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 15:50:14.032771 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026230 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026233 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026235 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026238 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026241 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026245 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026248 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:14.026251 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.026256 2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.026350 2547 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.028205 2547 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.029635 2547 server.go:1019] "Starting client certificate rotation" Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.029754 2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 15:50:14.033330 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.029803 2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 15:50:14.033656 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.031724 2547 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 15:50:14.033656 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.032857 2547 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 15:50:14.043164 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.043146 2547 log.go:25] "Validated CRI v1 runtime API" Jul 23 15:50:14.046119 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.046100 2547 log.go:25] "Validated CRI v1 image API" Jul 23 15:50:14.046970 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.046953 2547 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 15:50:14.049469 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.049449 2547 fs.go:135] Filesystem UUIDs: map[182ca147-e414-4bbb-80a9-b3f24be296be:/dev/nvme0n1p3 23e218ad-6ee7-4a65-97aa-42e07c8278b6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 23 15:50:14.049562 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.049467 2547 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 15:50:14.055240 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.055124 2547 manager.go:216] Machine: {Timestamp:2026-07-23 15:50:14.053938756 +0000 UTC m=+0.330680147 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100400 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec233d65ae614e23b93d48ca7a16eca7 SystemUUID:ec233d65-ae61-4e23-b93d-48ca7a16eca7 BootID:70bad63c-7953-46f3-b9ab-9ad4a7fe5d12 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:55:b3:e9:1b:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:55:b3:e9:1b:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:62:82:a8:6c:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 15:50:14.055240 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.055228 2547 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 15:50:14.055403 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.055330 2547 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 15:50:14.056554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.056526 2547 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 15:50:14.056754 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.056557 2547 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 15:50:14.057418 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.057406 2547 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 15:50:14.057468 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.057424 2547 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 15:50:14.057468 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.057435 2547 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 15:50:14.058097 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.058085 2547 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 15:50:14.059245 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.059233 2547 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 15:50:14.060341 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.060326 2547 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 15:50:14.062137 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062124 2547 state_mem.go:36] "Initialized new in-memory state store" Jul 23 15:50:14.062266 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062255 2547 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 15:50:14.062320 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062291 2547 kubelet.go:486] "Attempting to sync node with API server" Jul 23 15:50:14.062320 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062307 2547 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 15:50:14.062419 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062326 2547 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 15:50:14.062419 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062342 2547 kubelet.go:398] "Adding apiserver pod source" Jul 23 15:50:14.062419 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062369 2547 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 15:50:14.062758 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062744 2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 15:50:14.062834 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.062767 2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 15:50:14.065228 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.065214 2547 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 15:50:14.066528 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.066516 2547 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 15:50:14.066577 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.066545 2547 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 15:50:14.067070 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067060 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 15:50:14.067102 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067075 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 15:50:14.067102 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067081 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 15:50:14.067102 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067087 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 15:50:14.067102 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067095 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 15:50:14.067102 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067100 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 15:50:14.067230 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067132 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 15:50:14.067230 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067138 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 15:50:14.067230 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067145 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 15:50:14.067230 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067150 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 15:50:14.067230 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067158 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 15:50:14.067230 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067166 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 15:50:14.067745 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067735 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 15:50:14.067745 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.067744 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 15:50:14.070785 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.070773 2547 server.go:1268] "Started kubelet" Jul 23 15:50:14.070863 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.070843 2547 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 15:50:14.070941 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.070891 2547 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 15:50:14.070991 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.070967 2547 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 15:50:14.071630 ip-10-0-139-161 systemd[1]: Started Kubernetes Kubelet. Jul 23 15:50:14.073910 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.073888 2547 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 15:50:14.075395 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.075369 2547 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 15:50:14.075709 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.075678 2547 server.go:310] "Adding debug handlers to kubelet server" Jul 23 15:50:14.077764 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.077744 2547 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 15:50:14.077764 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.077747 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 15:50:14.077987 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.077969 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 15:50:14.079448 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.078475 2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-161.ec2.internal.18c4f5fbc70e216f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-161.ec2.internal,UID:ip-10-0-139-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-161.ec2.internal,},FirstTimestamp:2026-07-23 15:50:14.070747503 +0000 UTC m=+0.347488894,LastTimestamp:2026-07-23 15:50:14.070747503 +0000 UTC m=+0.347488894,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-161.ec2.internal,}" Jul 23 15:50:14.079877 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.079859 2547 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 15:50:14.079951 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.079874 2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 15:50:14.080004 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.079951 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.080447 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080358 2547 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 15:50:14.080447 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080359 2547 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 15:50:14.080447 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080396 2547 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 15:50:14.080671 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080505 2547 reconstruct.go:97] "Volume reconstruction finished" Jul 23 15:50:14.080671 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080515 2547 reconciler.go:29] "Reconciler: start to sync state" Jul 23 15:50:14.080792 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080676 2547 factory.go:55] Registering systemd factory Jul 23 15:50:14.080792 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080706 2547 factory.go:223] Registration of the systemd container factory successfully Jul 23 15:50:14.080973 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080958 2547 factory.go:153] Registering CRI-O factory Jul 23 15:50:14.081023 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.080975 2547 factory.go:223] Registration of the crio container factory successfully Jul 23 15:50:14.081057 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.081022 2547 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 15:50:14.081057 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.081046 2547 factory.go:103] Registering Raw factory Jul 23 15:50:14.081135 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.081062 2547 manager.go:1196] Started watching for new ooms in manager Jul 23 15:50:14.081729 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.081713 2547 manager.go:349] Starting recovery of all containers Jul 23 15:50:14.091743 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.091584 2547 manager.go:354] Recovery completed Jul 23 15:50:14.092787 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.092764 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 15:50:14.092787 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.092769 2547 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 15:50:14.096036 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.096023 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:14.097993 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.097967 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:14.098063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.098004 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:14.098063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.098015 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:14.098548 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.098533 2547 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 15:50:14.098548 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.098546 2547 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 15:50:14.098662 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.098559 2547 state_mem.go:36] "Initialized new in-memory state store" Jul 23 15:50:14.098662 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.098582 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:14.099496 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.099481 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:14.099552 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.099514 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:14.099552 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.099529 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:14.099656 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.099641 2547 policy_none.go:49] "None policy: Start" Jul 23 15:50:14.099656 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.099654 2547 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 15:50:14.099729 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.099664 2547 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 15:50:14.100660 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.100650 2547 policy_none.go:47] "Start" Jul 23 15:50:14.109461 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.109390 2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-161.ec2.internal.18c4f5fbc8add792 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-161.ec2.internal,UID:ip-10-0-139-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-161.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-161.ec2.internal,},FirstTimestamp:2026-07-23 15:50:14.09799157 +0000 UTC m=+0.374732965,LastTimestamp:2026-07-23 15:50:14.09799157 +0000 UTC m=+0.374732965,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-161.ec2.internal,}" Jul 23 15:50:14.114010 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.113990 2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-shqx5" Jul 23 15:50:14.119479 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.119413 2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-161.ec2.internal.18c4f5fbc8ae1d60 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-161.ec2.internal,UID:ip-10-0-139-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-161.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-161.ec2.internal,},FirstTimestamp:2026-07-23 15:50:14.09800944 +0000 UTC m=+0.374750831,LastTimestamp:2026-07-23 15:50:14.09800944 +0000 UTC m=+0.374750831,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-161.ec2.internal,}" Jul 23 15:50:14.120099 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.120073 2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-shqx5" Jul 23 15:50:14.134948 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.134934 2547 manager.go:337] "Starting Device Plugin manager" Jul 23 15:50:14.135020 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.134964 2547 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 15:50:14.135020 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.134974 2547 server.go:87] "Starting device plugin registration server" Jul 23 15:50:14.135206 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.135193 2547 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 15:50:14.135254 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.135208 2547 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 15:50:14.135337 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.135322 2547 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 15:50:14.135411 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.135399 2547 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 15:50:14.135411 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.135409 2547 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 15:50:14.135997 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.135974 2547 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 15:50:14.136095 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.136021 2547 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.233341 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.233302 2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 15:50:14.234449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.234433 2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 15:50:14.234537 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.234468 2547 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 15:50:14.234537 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.234490 2547 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 15:50:14.234537 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.234524 2547 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 15:50:14.235286 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.235269 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:14.236118 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.236095 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:14.236202 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.236134 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:14.236202 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.236150 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:14.236202 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.236181 2547 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.237054 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.237040 2547 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:50:14.246005 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.245990 2547 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.246071 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.246012 2547 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-161.ec2.internal\": node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.271027 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.270969 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.335384 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.335354 2547 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal"] Jul 23 15:50:14.335501 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.335421 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:14.336356 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.336332 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:14.336445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.336363 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:14.336445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.336377 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:14.337621 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.337610 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:14.337755 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.337738 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.337798 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.337787 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:14.338337 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.338321 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:14.338337 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.338331 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:14.338438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.338347 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:14.338438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.338349 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:14.338438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.338361 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:14.338524 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.338361 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:14.339403 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.339388 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.339484 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.339416 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:14.340041 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.340025 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:14.340134 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.340052 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:14.340134 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.340061 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:14.369412 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.369394 2547 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-161.ec2.internal\" not found" node="ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.371351 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.371334 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.373726 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.373709 2547 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-161.ec2.internal\" not found" node="ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.472159 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.472129 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.481485 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.481462 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe2cb68e1c0e89ce47bc3855be79f24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal\" (UID: \"7fe2cb68e1c0e89ce47bc3855be79f24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.481536 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.481502 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe2cb68e1c0e89ce47bc3855be79f24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal\" (UID: \"7fe2cb68e1c0e89ce47bc3855be79f24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.481536 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.481526 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d6081f961b13beba9ffc30484d6a818-config\") pod \"kube-apiserver-proxy-ip-10-0-139-161.ec2.internal\" (UID: \"9d6081f961b13beba9ffc30484d6a818\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.572944 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.572885 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.582174 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.582158 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe2cb68e1c0e89ce47bc3855be79f24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal\" (UID: \"7fe2cb68e1c0e89ce47bc3855be79f24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.582223 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.582183 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe2cb68e1c0e89ce47bc3855be79f24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal\" (UID: \"7fe2cb68e1c0e89ce47bc3855be79f24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.582223 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.582202 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d6081f961b13beba9ffc30484d6a818-config\") pod \"kube-apiserver-proxy-ip-10-0-139-161.ec2.internal\" (UID: \"9d6081f961b13beba9ffc30484d6a818\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.582286 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.582253 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d6081f961b13beba9ffc30484d6a818-config\") pod \"kube-apiserver-proxy-ip-10-0-139-161.ec2.internal\" (UID: \"9d6081f961b13beba9ffc30484d6a818\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.582286 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.582274 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe2cb68e1c0e89ce47bc3855be79f24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal\" (UID: \"7fe2cb68e1c0e89ce47bc3855be79f24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.582346 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.582256 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe2cb68e1c0e89ce47bc3855be79f24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal\" (UID: \"7fe2cb68e1c0e89ce47bc3855be79f24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.671348 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.671315 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.673314 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.673283 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.677949 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:14.677934 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" Jul 23 15:50:14.773447 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.773406 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.873960 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.873886 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:14.974382 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:14.974353 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:15.029943 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.029913 2547 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 15:50:15.030424 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.030065 2547 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 15:50:15.075405 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:15.075372 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:15.123493 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.123458 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 15:45:14 +0000 UTC" deadline="2028-02-13 11:45:02.960225101 +0000 UTC" Jul 23 15:50:15.123493 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.123490 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13675h54m47.836738103s" Jul 23 15:50:15.176107 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:15.176087 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:15.181019 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:15.180981 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6081f961b13beba9ffc30484d6a818.slice/crio-2520af1b9f94fcc1c14cd1793bccbc9944f25c19123839c041fa31d2a9674111 WatchSource:0}: Error finding container 2520af1b9f94fcc1c14cd1793bccbc9944f25c19123839c041fa31d2a9674111: Status 404 returned error can't find the container with id 2520af1b9f94fcc1c14cd1793bccbc9944f25c19123839c041fa31d2a9674111 Jul 23 15:50:15.185532 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.185520 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:50:15.185766 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.185752 2547 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:50:15.197335 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.197315 2547 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:50:15.238024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.237990 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" event={"ID":"9d6081f961b13beba9ffc30484d6a818","Type":"ContainerStarted","Data":"2520af1b9f94fcc1c14cd1793bccbc9944f25c19123839c041fa31d2a9674111"} Jul 23 15:50:15.277147 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:15.277129 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:15.279273 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.279262 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 15:50:15.280234 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.280214 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientMemory" Jul 23 15:50:15.280307 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.280246 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 15:50:15.280307 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.280260 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeHasSufficientPID" Jul 23 15:50:15.378128 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:15.378088 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:15.478626 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:15.478547 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-161.ec2.internal\" not found" Jul 23 15:50:15.515684 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.515655 2547 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:50:15.580122 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.580100 2547 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" Jul 23 15:50:15.593806 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.593780 2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 15:50:15.594802 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.594789 2547 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" Jul 23 15:50:15.603526 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:15.603510 2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 15:50:15.673645 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:15.673619 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe2cb68e1c0e89ce47bc3855be79f24.slice/crio-a9fccbe2efa1c7f5547ec85752168bd7092e303878d5c8442b22c61cae2e1a88 WatchSource:0}: Error finding container a9fccbe2efa1c7f5547ec85752168bd7092e303878d5c8442b22c61cae2e1a88: Status 404 returned error can't find the container with id a9fccbe2efa1c7f5547ec85752168bd7092e303878d5c8442b22c61cae2e1a88 Jul 23 15:50:16.063838 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.063807 2547 apiserver.go:52] "Watching apiserver" Jul 23 15:50:16.071004 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.070969 2547 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 15:50:16.071395 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.071367 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-kfk64","openshift-dns/node-resolver-rrxrl","openshift-image-registry/node-ca-lx88p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal","openshift-multus/multus-additional-cni-plugins-ql7xx","openshift-multus/multus-qvxz2","kube-system/konnectivity-agent-78ctg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8","openshift-multus/network-metrics-daemon-sg68n","openshift-network-diagnostics/network-check-target-tgdf5","openshift-network-operator/iptables-alerter-qfd7m","openshift-ovn-kubernetes/ovnkube-node-k4qjk","kube-system/global-pull-secret-syncer-gphkh","kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal"] Jul 23 15:50:16.072813 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.072791 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:16.072943 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.072873 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:16.074395 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.074378 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.074488 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.074445 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:16.075856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.075375 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.076852 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.076401 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.078388 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.077628 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.078388 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.077682 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zmdl4\"" Jul 23 15:50:16.078388 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.077982 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 15:50:16.078633 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.078450 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 15:50:16.078729 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.078658 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 15:50:16.078869 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.078846 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 15:50:16.078962 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.078900 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 15:50:16.079028 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.078906 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4zzrj\"" Jul 23 15:50:16.079161 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.079145 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.079355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.079227 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 15:50:16.081018 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.080458 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8mfh4\"" Jul 23 15:50:16.081018 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.080606 2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 15:50:16.081018 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.080652 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 15:50:16.081018 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.080833 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 15:50:16.081018 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.080890 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 15:50:16.081322 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.081066 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:16.081322 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.081129 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 15:50:16.081322 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.081134 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:16.081322 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.081190 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.081658 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.081609 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 15:50:16.082092 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.081837 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r7jtb\"" Jul 23 15:50:16.082092 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.081938 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 15:50:16.082843 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.082821 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.083872 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.083414 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.084108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.083998 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 15:50:16.084937 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.084527 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 15:50:16.084937 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.084594 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-92z6n\"" Jul 23 15:50:16.087445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.086668 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 15:50:16.087445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.086685 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 15:50:16.087445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.086668 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:50:16.087445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.086796 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8dfn9\"" Jul 23 15:50:16.087445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.086985 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-569r9\"" Jul 23 15:50:16.087752 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.087727 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.088589 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088568 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/029afce9-440f-4b8c-992f-97e2786871bf-host\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.088706 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088610 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.088706 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088643 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-registration-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.088706 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088670 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-cnibin\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.088706 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088678 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 15:50:16.088916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088720 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-os-release\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.088916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088752 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-k8s-cni-cncf-io\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.088916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088782 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-netns\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.088916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088811 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.088916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088845 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx97c\" (UniqueName: \"kubernetes.io/projected/029afce9-440f-4b8c-992f-97e2786871bf-kube-api-access-lx97c\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.088916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088879 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1e6c795-dcda-450f-b818-06d15f4a1a48-cni-binary-copy\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.089128 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.088967 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-kubelet-config\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.089181 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089159 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.089767 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089643 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.089767 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089678 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt7pb\" (UniqueName: \"kubernetes.io/projected/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-kube-api-access-mt7pb\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.089767 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089719 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48d96dfd-fc36-408b-bc0c-02b407eaa2d1-agent-certs\") pod \"konnectivity-agent-78ctg\" (UID: \"48d96dfd-fc36-408b-bc0c-02b407eaa2d1\") " pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.089767 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089746 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/029afce9-440f-4b8c-992f-97e2786871bf-serviceca\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089777 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-sys-fs\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089807 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-daemon-config\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089836 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5bwj\" (UniqueName: \"kubernetes.io/projected/d199f478-aba8-452c-9224-ed736a726a29-kube-api-access-w5bwj\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089863 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-dbus\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089886 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-system-cni-dir\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089918 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-os-release\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089941 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-device-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.089965 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-cni-multus\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090011 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc89z\" (UniqueName: \"kubernetes.io/projected/f1e6c795-dcda-450f-b818-06d15f4a1a48-kube-api-access-dc89z\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090034 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-socket-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090060 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-kubelet\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090088 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-etc-kubernetes\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090112 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48d96dfd-fc36-408b-bc0c-02b407eaa2d1-konnectivity-ca\") pod \"konnectivity-agent-78ctg\" (UID: \"48d96dfd-fc36-408b-bc0c-02b407eaa2d1\") " pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090135 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-system-cni-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090188 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-cni-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090210 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-hostroot\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090234 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cnibin\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090260 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090286 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090312 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zx6w\" (UniqueName: \"kubernetes.io/projected/c4375507-ff84-410a-bbf3-e8ceb03efdb8-kube-api-access-6zx6w\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090342 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-cni-bin\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090357 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090367 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090393 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.090445 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090419 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:16.091192 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090490 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-socket-dir-parent\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.091192 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090516 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-conf-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.091192 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090542 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-multus-certs\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.091192 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090825 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 15:50:16.091192 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.090890 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:50:16.091192 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.091033 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z2ndl\"" Jul 23 15:50:16.091472 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.091287 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 15:50:16.093218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.093199 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 15:50:16.093306 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.093238 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 15:50:16.093306 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.093247 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nsthw\"" Jul 23 15:50:16.094180 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.094140 2547 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 15:50:16.094267 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.094182 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 15:50:16.094267 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.094239 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 15:50:16.094541 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.094382 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 15:50:16.094541 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.094480 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 15:50:16.120539 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.120522 2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9rqz8" Jul 23 15:50:16.134644 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.134623 2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9rqz8" Jul 23 15:50:16.182045 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.182023 2547 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 15:50:16.191138 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191113 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx97c\" (UniqueName: \"kubernetes.io/projected/029afce9-440f-4b8c-992f-97e2786871bf-kube-api-access-lx97c\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.191232 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191151 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1e6c795-dcda-450f-b818-06d15f4a1a48-cni-binary-copy\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.191232 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191177 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-kubelet-config\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.191232 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191202 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/029afce9-440f-4b8c-992f-97e2786871bf-serviceca\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.191232 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191227 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-sys-fs\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.191438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191251 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-daemon-config\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.191438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191276 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5bwj\" (UniqueName: \"kubernetes.io/projected/d199f478-aba8-452c-9224-ed736a726a29-kube-api-access-w5bwj\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:16.191438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191367 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-sys-fs\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.191573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191473 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-kubelet-config\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.191573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191511 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-dbus\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.191573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191542 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.191573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191570 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-env-overrides\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191596 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-device-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191623 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhn9t\" (UniqueName: \"kubernetes.io/projected/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-kube-api-access-qhn9t\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191652 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-socket-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191676 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7d9a113-8cc2-42e8-807c-58db20ab3221-tmp-dir\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191666 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-dbus\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191686 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-device-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191720 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp5b8\" (UniqueName: \"kubernetes.io/projected/e7d9a113-8cc2-42e8-807c-58db20ab3221-kube-api-access-tp5b8\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191745 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysconfig\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191768 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-systemd\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191794 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-cni-bin\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.191823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191818 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191827 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-socket-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191837 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/029afce9-440f-4b8c-992f-97e2786871bf-serviceca\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191841 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191874 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-lib-modules\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191897 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gshw5\" (UniqueName: \"kubernetes.io/projected/87716055-18a5-41b3-8147-572541c957db-kube-api-access-gshw5\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191908 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-daemon-config\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.191929 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191972 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-cni-bin\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191928 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.192005 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs podName:d199f478-aba8-452c-9224-ed736a726a29 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:16.691969418 +0000 UTC m=+2.968710818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs") pod "network-metrics-daemon-sg68n" (UID: "d199f478-aba8-452c-9224-ed736a726a29") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.191998 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192093 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-socket-dir-parent\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192126 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-slash\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192148 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-socket-dir-parent\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192156 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-run-netns\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192181 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-etc-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.192333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192206 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovnkube-script-lib\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192237 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-registration-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192263 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-os-release\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192288 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-k8s-cni-cncf-io\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192294 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1e6c795-dcda-450f-b818-06d15f4a1a48-cni-binary-copy\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192298 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-registration-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192312 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8091f247-ab19-4134-ae7e-289cf19b1db8-iptables-alerter-script\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192363 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-ovn\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192369 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-k8s-cni-cncf-io\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192394 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-cni-netd\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192419 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7d9a113-8cc2-42e8-807c-58db20ab3221-hosts-file\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192428 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-os-release\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192462 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-var-lib-kubelet\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192486 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-var-lib-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192536 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192618 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt7pb\" (UniqueName: \"kubernetes.io/projected/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-kube-api-access-mt7pb\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.192976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.192647 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48d96dfd-fc36-408b-bc0c-02b407eaa2d1-agent-certs\") pod \"konnectivity-agent-78ctg\" (UID: \"48d96dfd-fc36-408b-bc0c-02b407eaa2d1\") " pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193070 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-host\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193069 2547 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193122 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-system-cni-dir\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193168 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-system-cni-dir\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193198 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-os-release\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193233 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-systemd\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193256 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovn-node-metrics-cert\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193291 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysctl-conf\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193318 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-cni-multus\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193323 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193344 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc89z\" (UniqueName: \"kubernetes.io/projected/f1e6c795-dcda-450f-b818-06d15f4a1a48-kube-api-access-dc89z\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193405 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-cni-multus\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193444 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-os-release\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193453 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193497 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-modprobe-d\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193522 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-kubelet\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.193668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193546 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-etc-kubernetes\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193569 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovnkube-config\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193610 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-etc-kubernetes\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193611 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-var-lib-kubelet\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193649 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-run\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193672 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87716055-18a5-41b3-8147-572541c957db-tmp\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193715 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-system-cni-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193742 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-cni-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193764 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-hostroot\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193785 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cnibin\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193810 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193835 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-node-log\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193859 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-cni-bin\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193876 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-system-cni-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193882 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193922 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zx6w\" (UniqueName: \"kubernetes.io/projected/c4375507-ff84-410a-bbf3-e8ceb03efdb8-kube-api-access-6zx6w\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193950 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8091f247-ab19-4134-ae7e-289cf19b1db8-host-slash\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.194452 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193973 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-etc-selinux\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193976 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj49n\" (UniqueName: \"kubernetes.io/projected/8091f247-ab19-4134-ae7e-289cf19b1db8-kube-api-access-jj49n\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194011 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-hostroot\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194036 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-kubelet\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194059 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysctl-d\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194082 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87716055-18a5-41b3-8147-572541c957db-etc-tuned\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194107 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-conf-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194141 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-multus-certs\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194166 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-run-ovn-kubernetes\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.193832 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-cni-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194201 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-kubernetes\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194246 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/029afce9-440f-4b8c-992f-97e2786871bf-host\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194252 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-multus-conf-dir\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194283 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194308 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cnibin\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194309 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-cnibin\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194285 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-multus-certs\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194351 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-cnibin\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194386 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-netns\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194411 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194438 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194446 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/029afce9-440f-4b8c-992f-97e2786871bf-host\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194469 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-log-socket\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194494 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-sys\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194503 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1e6c795-dcda-450f-b818-06d15f4a1a48-host-run-netns\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194533 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48d96dfd-fc36-408b-bc0c-02b407eaa2d1-konnectivity-ca\") pod \"konnectivity-agent-78ctg\" (UID: \"48d96dfd-fc36-408b-bc0c-02b407eaa2d1\") " pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194564 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-systemd-units\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194573 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.194591 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.194629 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4375507-ff84-410a-bbf3-e8ceb03efdb8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.194647 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret podName:b38c39bd-bd8b-40e9-95ad-c1ef17a463a0 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:16.69463058 +0000 UTC m=+2.971371960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret") pod "global-pull-secret-syncer-gphkh" (UID: "b38c39bd-bd8b-40e9-95ad-c1ef17a463a0") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.195104 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.195682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.195161 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48d96dfd-fc36-408b-bc0c-02b407eaa2d1-konnectivity-ca\") pod \"konnectivity-agent-78ctg\" (UID: \"48d96dfd-fc36-408b-bc0c-02b407eaa2d1\") " pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.197897 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.197857 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48d96dfd-fc36-408b-bc0c-02b407eaa2d1-agent-certs\") pod \"konnectivity-agent-78ctg\" (UID: \"48d96dfd-fc36-408b-bc0c-02b407eaa2d1\") " pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.198385 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.198367 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:50:16.198501 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.198390 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:50:16.198501 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.198402 2547 projected.go:196] Error preparing data for projected volume kube-api-access-mf69f for pod openshift-network-diagnostics/network-check-target-tgdf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:16.198501 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.198449 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f podName:e15e83d8-73c2-47c3-95f3-6763d5f5114f nodeName:}" failed. No retries permitted until 2026-07-23 15:50:16.698434581 +0000 UTC m=+2.975175966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mf69f" (UniqueName: "kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f") pod "network-check-target-tgdf5" (UID: "e15e83d8-73c2-47c3-95f3-6763d5f5114f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:16.202066 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.202014 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx97c\" (UniqueName: \"kubernetes.io/projected/029afce9-440f-4b8c-992f-97e2786871bf-kube-api-access-lx97c\") pod \"node-ca-lx88p\" (UID: \"029afce9-440f-4b8c-992f-97e2786871bf\") " pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.203141 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.203116 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zx6w\" (UniqueName: \"kubernetes.io/projected/c4375507-ff84-410a-bbf3-e8ceb03efdb8-kube-api-access-6zx6w\") pod \"aws-ebs-csi-driver-node-p94z8\" (UID: \"c4375507-ff84-410a-bbf3-e8ceb03efdb8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.203650 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.203627 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt7pb\" (UniqueName: \"kubernetes.io/projected/f76a8e9b-f800-4ecc-ad18-08d9df0c8096-kube-api-access-mt7pb\") pod \"multus-additional-cni-plugins-ql7xx\" (UID: \"f76a8e9b-f800-4ecc-ad18-08d9df0c8096\") " pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.203776 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.203701 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5bwj\" (UniqueName: \"kubernetes.io/projected/d199f478-aba8-452c-9224-ed736a726a29-kube-api-access-w5bwj\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:16.205485 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.205463 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc89z\" (UniqueName: \"kubernetes.io/projected/f1e6c795-dcda-450f-b818-06d15f4a1a48-kube-api-access-dc89z\") pod \"multus-qvxz2\" (UID: \"f1e6c795-dcda-450f-b818-06d15f4a1a48\") " pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.240548 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.240511 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" event={"ID":"7fe2cb68e1c0e89ce47bc3855be79f24","Type":"ContainerStarted","Data":"a9fccbe2efa1c7f5547ec85752168bd7092e303878d5c8442b22c61cae2e1a88"} Jul 23 15:50:16.294951 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.294914 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-slash\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.294959 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-run-netns\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.294984 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-etc-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295006 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovnkube-script-lib\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295013 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-slash\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295031 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8091f247-ab19-4134-ae7e-289cf19b1db8-iptables-alerter-script\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295057 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-ovn\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295062 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-etc-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295081 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-cni-netd\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295102 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-run-netns\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295105 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7d9a113-8cc2-42e8-807c-58db20ab3221-hosts-file\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295128 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-var-lib-kubelet\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295127 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-ovn\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295151 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-var-lib-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295161 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-cni-netd\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295177 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-host\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295201 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e7d9a113-8cc2-42e8-807c-58db20ab3221-hosts-file\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295210 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-var-lib-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295217 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-systemd\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295222 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-host\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295246 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-var-lib-kubelet\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295289 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovn-node-metrics-cert\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295315 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysctl-conf\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295323 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-systemd\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295434 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295483 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295514 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-modprobe-d\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295516 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysctl-conf\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295591 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovnkube-config\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295594 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-modprobe-d\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295624 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-run\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295649 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87716055-18a5-41b3-8147-572541c957db-tmp\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295676 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-node-log\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.295866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295680 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-run\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295732 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovnkube-script-lib\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295786 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-cni-bin\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295802 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-node-log\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295785 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8091f247-ab19-4134-ae7e-289cf19b1db8-iptables-alerter-script\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295816 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8091f247-ab19-4134-ae7e-289cf19b1db8-host-slash\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295832 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-cni-bin\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295855 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8091f247-ab19-4134-ae7e-289cf19b1db8-host-slash\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295866 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj49n\" (UniqueName: \"kubernetes.io/projected/8091f247-ab19-4134-ae7e-289cf19b1db8-kube-api-access-jj49n\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295891 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-kubelet\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295914 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysctl-d\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295936 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87716055-18a5-41b3-8147-572541c957db-etc-tuned\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295960 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-run-ovn-kubernetes\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295983 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-kubernetes\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296023 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-log-socket\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296045 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-sys\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296048 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysctl-d\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.296665 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296088 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-run-ovn-kubernetes\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296094 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-sys\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.295961 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-host-kubelet\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296125 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-systemd-units\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296149 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-kubernetes\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296160 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296140 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-log-socket\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296162 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovnkube-config\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296194 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-systemd-units\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296205 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-run-openvswitch\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296237 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-env-overrides\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296262 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn9t\" (UniqueName: \"kubernetes.io/projected/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-kube-api-access-qhn9t\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296303 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7d9a113-8cc2-42e8-807c-58db20ab3221-tmp-dir\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296325 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp5b8\" (UniqueName: \"kubernetes.io/projected/e7d9a113-8cc2-42e8-807c-58db20ab3221-kube-api-access-tp5b8\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296350 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysconfig\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296373 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-systemd\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296408 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-lib-modules\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296450 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gshw5\" (UniqueName: \"kubernetes.io/projected/87716055-18a5-41b3-8147-572541c957db-kube-api-access-gshw5\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.297449 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296570 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-sysconfig\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.298242 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296589 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7d9a113-8cc2-42e8-807c-58db20ab3221-tmp-dir\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.298242 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296624 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-env-overrides\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.298242 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296670 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-etc-systemd\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.298242 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.296738 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87716055-18a5-41b3-8147-572541c957db-lib-modules\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.298684 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.298664 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/87716055-18a5-41b3-8147-572541c957db-etc-tuned\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.298771 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.298724 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87716055-18a5-41b3-8147-572541c957db-tmp\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.299655 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.299633 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-ovn-node-metrics-cert\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.305548 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.305223 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj49n\" (UniqueName: \"kubernetes.io/projected/8091f247-ab19-4134-ae7e-289cf19b1db8-kube-api-access-jj49n\") pod \"iptables-alerter-qfd7m\" (UID: \"8091f247-ab19-4134-ae7e-289cf19b1db8\") " pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.305548 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.305455 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gshw5\" (UniqueName: \"kubernetes.io/projected/87716055-18a5-41b3-8147-572541c957db-kube-api-access-gshw5\") pod \"tuned-kfk64\" (UID: \"87716055-18a5-41b3-8147-572541c957db\") " pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.305758 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.305620 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp5b8\" (UniqueName: \"kubernetes.io/projected/e7d9a113-8cc2-42e8-807c-58db20ab3221-kube-api-access-tp5b8\") pod \"node-resolver-rrxrl\" (UID: \"e7d9a113-8cc2-42e8-807c-58db20ab3221\") " pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.307176 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.307152 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhn9t\" (UniqueName: \"kubernetes.io/projected/c102b23d-0f75-4ac7-b4d2-d7ba8f48a734-kube-api-access-qhn9t\") pod \"ovnkube-node-k4qjk\" (UID: \"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.389456 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.389372 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lx88p" Jul 23 15:50:16.403899 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.403868 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" Jul 23 15:50:16.412568 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.412540 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" Jul 23 15:50:16.419410 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.419387 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qvxz2" Jul 23 15:50:16.427051 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.427023 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:16.434907 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.434883 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kfk64" Jul 23 15:50:16.441749 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.441725 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rrxrl" Jul 23 15:50:16.450226 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.450202 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfd7m" Jul 23 15:50:16.457136 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.457110 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:16.551324 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.551293 2547 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 15:50:16.698906 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.698823 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:16.698906 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.698869 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:16.698927 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.698971 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.699028 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.699031 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.699054 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs podName:d199f478-aba8-452c-9224-ed736a726a29 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:17.699011638 +0000 UTC m=+3.975753020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs") pod "network-metrics-daemon-sg68n" (UID: "d199f478-aba8-452c-9224-ed736a726a29") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.699055 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.699076 2547 projected.go:196] Error preparing data for projected volume kube-api-access-mf69f for pod openshift-network-diagnostics/network-check-target-tgdf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:16.699106 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.699080 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret podName:b38c39bd-bd8b-40e9-95ad-c1ef17a463a0 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:17.699064589 +0000 UTC m=+3.975805982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret") pod "global-pull-secret-syncer-gphkh" (UID: "b38c39bd-bd8b-40e9-95ad-c1ef17a463a0") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:16.699422 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:16.699112 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f podName:e15e83d8-73c2-47c3-95f3-6763d5f5114f nodeName:}" failed. No retries permitted until 2026-07-23 15:50:17.699099938 +0000 UTC m=+3.975841318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf69f" (UniqueName: "kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f") pod "network-check-target-tgdf5" (UID: "e15e83d8-73c2-47c3-95f3-6763d5f5114f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:17.137072 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.136977 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 15:45:16 +0000 UTC" deadline="2028-02-04 04:50:21.997040955 +0000 UTC" Jul 23 15:50:17.137072 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.137020 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13453h0m4.860026061s" Jul 23 15:50:17.198066 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:17.198034 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4375507_ff84_410a_bbf3_e8ceb03efdb8.slice/crio-3e690b372395666bc3ec62521c88b7607c093ac2f4cf3f98d94b3108c27e1238 WatchSource:0}: Error finding container 3e690b372395666bc3ec62521c88b7607c093ac2f4cf3f98d94b3108c27e1238: Status 404 returned error can't find the container with id 3e690b372395666bc3ec62521c88b7607c093ac2f4cf3f98d94b3108c27e1238 Jul 23 15:50:17.199321 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:17.199298 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87716055_18a5_41b3_8147_572541c957db.slice/crio-64d5f76cdebd2d20188f6e73c38acc3f98f9dee9546a9535f1d3cebebc7704f5 WatchSource:0}: Error finding container 64d5f76cdebd2d20188f6e73c38acc3f98f9dee9546a9535f1d3cebebc7704f5: Status 404 returned error can't find the container with id 64d5f76cdebd2d20188f6e73c38acc3f98f9dee9546a9535f1d3cebebc7704f5 Jul 23 15:50:17.207408 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:17.206528 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76a8e9b_f800_4ecc_ad18_08d9df0c8096.slice/crio-c0f8a423f6ec31355c657ee3eeeecd27d15417e8e4ac343a2fad84c4086abe04 WatchSource:0}: Error finding container c0f8a423f6ec31355c657ee3eeeecd27d15417e8e4ac343a2fad84c4086abe04: Status 404 returned error can't find the container with id c0f8a423f6ec31355c657ee3eeeecd27d15417e8e4ac343a2fad84c4086abe04 Jul 23 15:50:17.208768 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:17.208742 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029afce9_440f_4b8c_992f_97e2786871bf.slice/crio-e1cc25aa9d30e7bfdde8ee7edd129e74bcb773d90b19d8e483786661ba7f42cd WatchSource:0}: Error finding container e1cc25aa9d30e7bfdde8ee7edd129e74bcb773d90b19d8e483786661ba7f42cd: Status 404 returned error can't find the container with id e1cc25aa9d30e7bfdde8ee7edd129e74bcb773d90b19d8e483786661ba7f42cd Jul 23 15:50:17.209776 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:17.209748 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d96dfd_fc36_408b_bc0c_02b407eaa2d1.slice/crio-7f885c41911e4cc44dd40e6533d60b49edb23563ccab5b63b35757dae9b6dc01 WatchSource:0}: Error finding container 7f885c41911e4cc44dd40e6533d60b49edb23563ccab5b63b35757dae9b6dc01: Status 404 returned error can't find the container with id 7f885c41911e4cc44dd40e6533d60b49edb23563ccab5b63b35757dae9b6dc01 Jul 23 15:50:17.210368 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:17.210345 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8091f247_ab19_4134_ae7e_289cf19b1db8.slice/crio-84a3d3b3b9b462310ba82be3ad0af9f8e8914ee045de0175b1e7718a988133f6 WatchSource:0}: Error finding container 84a3d3b3b9b462310ba82be3ad0af9f8e8914ee045de0175b1e7718a988133f6: Status 404 returned error can't find the container with id 84a3d3b3b9b462310ba82be3ad0af9f8e8914ee045de0175b1e7718a988133f6 Jul 23 15:50:17.211520 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:17.211146 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc102b23d_0f75_4ac7_b4d2_d7ba8f48a734.slice/crio-89248831c3391156e3849c5210842e0bd5cf564798763678c0b8e7e3e868f0e1 WatchSource:0}: Error finding container 89248831c3391156e3849c5210842e0bd5cf564798763678c0b8e7e3e868f0e1: Status 404 returned error can't find the container with id 89248831c3391156e3849c5210842e0bd5cf564798763678c0b8e7e3e868f0e1 Jul 23 15:50:17.235234 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.235205 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:17.235364 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.235257 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:17.235364 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.235347 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:17.235446 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.235422 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:17.245479 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.245448 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qvxz2" event={"ID":"f1e6c795-dcda-450f-b818-06d15f4a1a48","Type":"ContainerStarted","Data":"bbb1d5c77cb26cc1a1705228ec1183cf8516172e9f18989c97fd1a310cca57b0"} Jul 23 15:50:17.246657 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.246630 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lx88p" event={"ID":"029afce9-440f-4b8c-992f-97e2786871bf","Type":"ContainerStarted","Data":"e1cc25aa9d30e7bfdde8ee7edd129e74bcb773d90b19d8e483786661ba7f42cd"} Jul 23 15:50:17.247713 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.247671 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerStarted","Data":"c0f8a423f6ec31355c657ee3eeeecd27d15417e8e4ac343a2fad84c4086abe04"} Jul 23 15:50:17.248657 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.248640 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" event={"ID":"c4375507-ff84-410a-bbf3-e8ceb03efdb8","Type":"ContainerStarted","Data":"3e690b372395666bc3ec62521c88b7607c093ac2f4cf3f98d94b3108c27e1238"} Jul 23 15:50:17.251293 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.251272 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"89248831c3391156e3849c5210842e0bd5cf564798763678c0b8e7e3e868f0e1"} Jul 23 15:50:17.254603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.254577 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfd7m" event={"ID":"8091f247-ab19-4134-ae7e-289cf19b1db8","Type":"ContainerStarted","Data":"84a3d3b3b9b462310ba82be3ad0af9f8e8914ee045de0175b1e7718a988133f6"} Jul 23 15:50:17.255615 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.255585 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-78ctg" event={"ID":"48d96dfd-fc36-408b-bc0c-02b407eaa2d1","Type":"ContainerStarted","Data":"7f885c41911e4cc44dd40e6533d60b49edb23563ccab5b63b35757dae9b6dc01"} Jul 23 15:50:17.256518 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.256488 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kfk64" event={"ID":"87716055-18a5-41b3-8147-572541c957db","Type":"ContainerStarted","Data":"64d5f76cdebd2d20188f6e73c38acc3f98f9dee9546a9535f1d3cebebc7704f5"} Jul 23 15:50:17.257462 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.257440 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rrxrl" event={"ID":"e7d9a113-8cc2-42e8-807c-58db20ab3221","Type":"ContainerStarted","Data":"dad1c978e90fd5e88a05c09a736353ad35c4c7319ba405f08bb253394732cd48"} Jul 23 15:50:17.707203 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.707117 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:17.707203 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.707162 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:17.707203 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:17.707181 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707262 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707279 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707282 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707321 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret podName:b38c39bd-bd8b-40e9-95ad-c1ef17a463a0 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:19.707304789 +0000 UTC m=+5.984046167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret") pod "global-pull-secret-syncer-gphkh" (UID: "b38c39bd-bd8b-40e9-95ad-c1ef17a463a0") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707292 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707336 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs podName:d199f478-aba8-452c-9224-ed736a726a29 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:19.707328137 +0000 UTC m=+5.984069517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs") pod "network-metrics-daemon-sg68n" (UID: "d199f478-aba8-452c-9224-ed736a726a29") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707341 2547 projected.go:196] Error preparing data for projected volume kube-api-access-mf69f for pod openshift-network-diagnostics/network-check-target-tgdf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:17.707457 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:17.707391 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f podName:e15e83d8-73c2-47c3-95f3-6763d5f5114f nodeName:}" failed. No retries permitted until 2026-07-23 15:50:19.707373182 +0000 UTC m=+5.984114561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf69f" (UniqueName: "kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f") pod "network-check-target-tgdf5" (UID: "e15e83d8-73c2-47c3-95f3-6763d5f5114f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:18.137776 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:18.137587 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 15:45:16 +0000 UTC" deadline="2027-12-29 06:33:10.03977799 +0000 UTC" Jul 23 15:50:18.137776 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:18.137645 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12566h42m51.902137186s" Jul 23 15:50:18.237489 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:18.237035 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:18.237489 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:18.237151 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:18.265235 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:18.265203 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" event={"ID":"9d6081f961b13beba9ffc30484d6a818","Type":"ContainerStarted","Data":"55b8872030c16bbf7712d033630372c0df0a6a51e80a30ad256eb3f585f14078"} Jul 23 15:50:18.270900 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:18.270868 2547 generic.go:356] "Generic (PLEG): container finished" podID="7fe2cb68e1c0e89ce47bc3855be79f24" containerID="89a9d6b6c1e6639d5149cf848ae4f5ac564aa7a9f8883dda1a9597d9f45a6725" exitCode=0 Jul 23 15:50:18.271026 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:18.270915 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" event={"ID":"7fe2cb68e1c0e89ce47bc3855be79f24","Type":"ContainerDied","Data":"89a9d6b6c1e6639d5149cf848ae4f5ac564aa7a9f8883dda1a9597d9f45a6725"} Jul 23 15:50:18.300772 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:18.300380 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-161.ec2.internal" podStartSLOduration=3.300358859 podStartE2EDuration="3.300358859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:18.283071424 +0000 UTC m=+4.559812827" watchObservedRunningTime="2026-07-23 15:50:18.300358859 +0000 UTC m=+4.577100263" Jul 23 15:50:19.234998 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:19.234965 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:19.235601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:19.234982 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:19.235990 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.235937 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:19.236127 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.235855 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:19.298292 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:19.298226 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" event={"ID":"7fe2cb68e1c0e89ce47bc3855be79f24","Type":"ContainerStarted","Data":"440d7568f782bb6d621e84b9580143e8bfaeec44c4176643a46eda2250fdf59e"} Jul 23 15:50:19.725898 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:19.725798 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:19.725898 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:19.725875 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:19.726119 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:19.725906 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:19.726119 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726070 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:50:19.726119 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726092 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:50:19.726119 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726104 2547 projected.go:196] Error preparing data for projected volume kube-api-access-mf69f for pod openshift-network-diagnostics/network-check-target-tgdf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:19.726317 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726167 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f podName:e15e83d8-73c2-47c3-95f3-6763d5f5114f nodeName:}" failed. No retries permitted until 2026-07-23 15:50:23.726146664 +0000 UTC m=+10.002888046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf69f" (UniqueName: "kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f") pod "network-check-target-tgdf5" (UID: "e15e83d8-73c2-47c3-95f3-6763d5f5114f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:19.726715 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726552 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:19.726715 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726605 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret podName:b38c39bd-bd8b-40e9-95ad-c1ef17a463a0 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:23.726590333 +0000 UTC m=+10.003331718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret") pod "global-pull-secret-syncer-gphkh" (UID: "b38c39bd-bd8b-40e9-95ad-c1ef17a463a0") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:19.726715 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726660 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:19.726715 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:19.726708 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs podName:d199f478-aba8-452c-9224-ed736a726a29 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:23.726680451 +0000 UTC m=+10.003421831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs") pod "network-metrics-daemon-sg68n" (UID: "d199f478-aba8-452c-9224-ed736a726a29") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:20.235277 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:20.235242 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:20.235774 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:20.235382 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:21.236267 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:21.235662 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:21.236267 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:21.235718 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:21.236267 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:21.235825 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:21.236267 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:21.236228 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:22.235148 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:22.235111 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:22.235326 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:22.235270 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:23.235599 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:23.235570 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:23.236088 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.235706 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:23.236088 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:23.236055 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:23.236202 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.236181 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:23.763259 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:23.763217 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:23.763441 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:23.763281 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:23.763441 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:23.763308 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:23.763565 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763442 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:50:23.763565 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763469 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:23.763565 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763482 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:23.763565 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763549 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs podName:d199f478-aba8-452c-9224-ed736a726a29 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:31.763526424 +0000 UTC m=+18.040267816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs") pod "network-metrics-daemon-sg68n" (UID: "d199f478-aba8-452c-9224-ed736a726a29") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:23.763565 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763568 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret podName:b38c39bd-bd8b-40e9-95ad-c1ef17a463a0 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:31.763558317 +0000 UTC m=+18.040299699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret") pod "global-pull-secret-syncer-gphkh" (UID: "b38c39bd-bd8b-40e9-95ad-c1ef17a463a0") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:23.763873 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763477 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:50:23.763873 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763588 2547 projected.go:196] Error preparing data for projected volume kube-api-access-mf69f for pod openshift-network-diagnostics/network-check-target-tgdf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:23.763873 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:23.763623 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f podName:e15e83d8-73c2-47c3-95f3-6763d5f5114f nodeName:}" failed. No retries permitted until 2026-07-23 15:50:31.76361339 +0000 UTC m=+18.040354774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf69f" (UniqueName: "kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f") pod "network-check-target-tgdf5" (UID: "e15e83d8-73c2-47c3-95f3-6763d5f5114f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:24.236134 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:24.236019 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:24.236579 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:24.236138 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:25.235867 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:25.235397 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:25.235867 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:25.235515 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:25.235867 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:25.235755 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:25.235867 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:25.235868 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:26.234974 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:26.234939 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:26.235631 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:26.235572 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:27.235030 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:27.234981 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:27.235419 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:27.234989 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:27.235419 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:27.235093 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:27.235419 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:27.235176 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:28.235402 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:28.235363 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:28.235895 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:28.235503 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:29.234965 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:29.234926 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:29.235131 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:29.234967 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:29.235131 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:29.235054 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:29.235234 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:29.235178 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:30.237902 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:30.237869 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:30.238349 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:30.237976 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:31.235103 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:31.235076 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:31.235287 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:31.235076 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:31.235287 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.235203 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:31.235728 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.235687 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:31.826660 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:31.826622 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:31.826660 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:31.826661 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.826854 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.826874 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:31.826872 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.826880 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.826944 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.826884 2547 projected.go:196] Error preparing data for projected volume kube-api-access-mf69f for pod openshift-network-diagnostics/network-check-target-tgdf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.827001 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret podName:b38c39bd-bd8b-40e9-95ad-c1ef17a463a0 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:47.826982363 +0000 UTC m=+34.103723742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret") pod "global-pull-secret-syncer-gphkh" (UID: "b38c39bd-bd8b-40e9-95ad-c1ef17a463a0") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.827041 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs podName:d199f478-aba8-452c-9224-ed736a726a29 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:47.827029867 +0000 UTC m=+34.103771252 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs") pod "network-metrics-daemon-sg68n" (UID: "d199f478-aba8-452c-9224-ed736a726a29") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:31.827179 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:31.827054 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f podName:e15e83d8-73c2-47c3-95f3-6763d5f5114f nodeName:}" failed. No retries permitted until 2026-07-23 15:50:47.827046356 +0000 UTC m=+34.103787741 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf69f" (UniqueName: "kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f") pod "network-check-target-tgdf5" (UID: "e15e83d8-73c2-47c3-95f3-6763d5f5114f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:32.235080 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:32.234996 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:32.235240 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:32.235128 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:33.234875 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:33.234839 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:33.235312 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:33.234879 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:33.235312 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:33.234979 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:33.235312 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:33.235120 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:34.236238 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:34.236206 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:34.236646 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:34.236324 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:35.234884 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:35.234849 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:35.235061 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:35.234982 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:35.235061 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:35.235037 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:35.235164 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:35.235147 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:36.235818 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.235662 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:36.236269 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:36.235905 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:36.333492 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.333342 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lx88p" event={"ID":"029afce9-440f-4b8c-992f-97e2786871bf","Type":"ContainerStarted","Data":"74e70a6cb305b65bfa719a02850e5693d7ee68522266b6cd29ce2c587e2b04af"} Jul 23 15:50:36.335017 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.334992 2547 generic.go:356] "Generic (PLEG): container finished" podID="f76a8e9b-f800-4ecc-ad18-08d9df0c8096" containerID="c0626f36a1a996efe90badf47cca5c36e70c230dff11290ba359a3f0d1f7f267" exitCode=0 Jul 23 15:50:36.335108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.335069 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerDied","Data":"c0626f36a1a996efe90badf47cca5c36e70c230dff11290ba359a3f0d1f7f267"} Jul 23 15:50:36.336614 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.336592 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" event={"ID":"c4375507-ff84-410a-bbf3-e8ceb03efdb8","Type":"ContainerStarted","Data":"c1584c6e71aabfc66e50aa9a3ce20d625d6eceb70eca04f8d734c5e5fd92f93f"} Jul 23 15:50:36.342783 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.342761 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 15:50:36.343119 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.343098 2547 generic.go:356] "Generic (PLEG): container finished" podID="c102b23d-0f75-4ac7-b4d2-d7ba8f48a734" containerID="d91643a5622522812e9f9c58b574b789b3b4929eb0e4df85959ae598d0ac9c93" exitCode=1 Jul 23 15:50:36.343188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.343166 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"8b10b5041e0c80b9256dcb9557200fe6e266159e049c6b739a070746511b8eed"} Jul 23 15:50:36.343246 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.343234 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerDied","Data":"d91643a5622522812e9f9c58b574b789b3b4929eb0e4df85959ae598d0ac9c93"} Jul 23 15:50:36.343280 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.343269 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"2c066d9e01576ce44a52e33eb46bc51a62db62ef79e4b58ac5856156250dbde8"} Jul 23 15:50:36.344682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.344655 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-78ctg" event={"ID":"48d96dfd-fc36-408b-bc0c-02b407eaa2d1","Type":"ContainerStarted","Data":"aa2a0d8ea38d3070efaed0e909abcbfb83535ecd179bd85a41a722b72445d37d"} Jul 23 15:50:36.346205 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.346187 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kfk64" event={"ID":"87716055-18a5-41b3-8147-572541c957db","Type":"ContainerStarted","Data":"b80c2abc4c739032570179b28a48949377237857687247339851430ab98b7f45"} Jul 23 15:50:36.347430 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.347413 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rrxrl" event={"ID":"e7d9a113-8cc2-42e8-807c-58db20ab3221","Type":"ContainerStarted","Data":"18d4b296e7bd2d066e46643fd0a81bf5c456d61667ce8e87a93913a999e7e7cd"} Jul 23 15:50:36.349013 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.348978 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lx88p" podStartSLOduration=3.609266325 podStartE2EDuration="22.348968024s" totalImagesPullingTime="18.739701699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:36.348885052 +0000 UTC m=+22.625626453" watchObservedRunningTime="2026-07-23 15:50:36.348968024 +0000 UTC m=+22.625709424" Jul 23 15:50:36.349106 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.349041 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-161.ec2.internal" podStartSLOduration=21.349036059 podStartE2EDuration="21.349036059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:19.315907868 +0000 UTC m=+5.592649272" watchObservedRunningTime="2026-07-23 15:50:36.349036059 +0000 UTC m=+22.625777460" Jul 23 15:50:36.389966 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.389925 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-78ctg" podStartSLOduration=11.877998882 podStartE2EDuration="22.389910689s" totalImagesPullingTime="10.511911807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:36.389639949 +0000 UTC m=+22.666381352" watchObservedRunningTime="2026-07-23 15:50:36.389910689 +0000 UTC m=+22.666652090" Jul 23 15:50:36.405021 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.404981 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rrxrl" podStartSLOduration=3.661145631 podStartE2EDuration="22.404965764s" totalImagesPullingTime="18.743820133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:36.404750495 +0000 UTC m=+22.681491896" watchObservedRunningTime="2026-07-23 15:50:36.404965764 +0000 UTC m=+22.681707165" Jul 23 15:50:36.420317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:36.420276 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kfk64" podStartSLOduration=3.662115284 podStartE2EDuration="22.420258613s" totalImagesPullingTime="18.758143329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:36.419626017 +0000 UTC m=+22.696367430" watchObservedRunningTime="2026-07-23 15:50:36.420258613 +0000 UTC m=+22.697000013" Jul 23 15:50:37.234827 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.234791 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:37.235030 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:37.234900 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:37.235030 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.234954 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:37.235149 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:37.235085 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:37.352598 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.352574 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 15:50:37.353207 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.353000 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"d9adf4ff01101a092b05d73c0cfb3713462ddcd8bd4507ffcdcfa2d3a96f4224"} Jul 23 15:50:37.353207 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.353084 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"0edd413a20fa4c434fcb0a0905f9477e3e0480886192d732fd5d7fa42c7190a5"} Jul 23 15:50:37.353207 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.353131 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"287927f5d8f865805cfa08ccd0cfeb8e2c1f2dc41715e9a8fbdbb81aaa294c26"} Jul 23 15:50:37.354948 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.354921 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfd7m" event={"ID":"8091f247-ab19-4134-ae7e-289cf19b1db8","Type":"ContainerStarted","Data":"c65939e2203e17a674e31695d6af6f82a14a8e05b2e144f4c9e8008ac35c58b6"} Jul 23 15:50:37.356576 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.356553 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qvxz2" event={"ID":"f1e6c795-dcda-450f-b818-06d15f4a1a48","Type":"ContainerStarted","Data":"7e2822f29dc68f17e7667a8782145d84492f1ef85307077aba39b4f77ea17ac6"} Jul 23 15:50:37.370256 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.370216 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qfd7m" podStartSLOduration=4.63243651 podStartE2EDuration="23.370204399s" totalImagesPullingTime="18.737767889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:37.369970191 +0000 UTC m=+23.646711594" watchObservedRunningTime="2026-07-23 15:50:37.370204399 +0000 UTC m=+23.646945812" Jul 23 15:50:37.389280 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.389232 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qvxz2" podStartSLOduration=4.303861553 podStartE2EDuration="23.389215302s" totalImagesPullingTime="19.085353749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:37.389075116 +0000 UTC m=+23.665816518" watchObservedRunningTime="2026-07-23 15:50:37.389215302 +0000 UTC m=+23.665956702" Jul 23 15:50:37.578505 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:37.578480 2547 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 15:50:38.013599 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.013567 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:38.014346 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.014323 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:38.149174 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.149080 2547 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T15:50:37.578500635Z","UUID":"eb4be352-d455-437d-8ffb-91048ea2c65a","Handler":null,"Name":"","Endpoint":""} Jul 23 15:50:38.150663 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.150643 2547 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 15:50:38.150784 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.150686 2547 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 15:50:38.235077 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.235047 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:38.235241 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:38.235165 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:38.362940 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.362900 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" event={"ID":"c4375507-ff84-410a-bbf3-e8ceb03efdb8","Type":"ContainerStarted","Data":"bcfab72825f20a0d82214756c4a7e421d9a17468609a383089242ebbbb51e4f8"} Jul 23 15:50:38.363431 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.363398 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:38.363796 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:38.363776 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-78ctg" Jul 23 15:50:39.235758 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:39.235500 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:39.235758 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:39.235500 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:39.235987 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:39.235835 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:39.235987 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:39.235938 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:39.368842 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:39.368812 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 15:50:39.369292 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:39.369261 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"659b73d1e871ed6b316b4841112a2e5b2be3a9e1d8ac1d2b48e2ebbd20b20fe7"} Jul 23 15:50:40.235768 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:40.235732 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:40.235939 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:40.235870 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:41.235569 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.235349 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:41.236248 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.235352 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:41.236248 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:41.235576 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:41.236248 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:41.235668 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:41.375197 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.375166 2547 generic.go:356] "Generic (PLEG): container finished" podID="f76a8e9b-f800-4ecc-ad18-08d9df0c8096" containerID="833ba126c197bb694ca9091c108ae038180c472552e9e2df825b3807059e33d7" exitCode=0 Jul 23 15:50:41.375349 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.375239 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerDied","Data":"833ba126c197bb694ca9091c108ae038180c472552e9e2df825b3807059e33d7"} Jul 23 15:50:41.377419 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.377397 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" event={"ID":"c4375507-ff84-410a-bbf3-e8ceb03efdb8","Type":"ContainerStarted","Data":"58663bae979253da224bacbcb4aec83d6bb7e8f967c2d6c86c91cdb6812649f6"} Jul 23 15:50:41.380856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.380838 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 15:50:41.381168 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.381149 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"9826ba26defb052747a7c3dbee8edfc9e8550ecc2fab0d765ab713b91c1abd89"} Jul 23 15:50:41.381408 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.381393 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:41.381483 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.381448 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:41.381483 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.381480 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:41.381576 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.381562 2547 scope.go:117] "RemoveContainer" containerID="d91643a5622522812e9f9c58b574b789b3b4929eb0e4df85959ae598d0ac9c93" Jul 23 15:50:41.397409 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.397390 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:41.397621 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.397603 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:50:41.439064 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:41.439024 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p94z8" podStartSLOduration=4.459391306 podStartE2EDuration="27.439009628s" totalImagesPullingTime="22.979618322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:41.413743283 +0000 UTC m=+27.690484684" watchObservedRunningTime="2026-07-23 15:50:41.439009628 +0000 UTC m=+27.715751028" Jul 23 15:50:42.235025 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.234944 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:42.235206 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:42.235086 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:42.387503 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.387476 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 15:50:42.387931 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.387817 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" event={"ID":"c102b23d-0f75-4ac7-b4d2-d7ba8f48a734","Type":"ContainerStarted","Data":"2c741e655ab1f6f9422f8b508eaf1bf16af8d14967893c0aca19f1231adeb483"} Jul 23 15:50:42.415293 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.415247 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" podStartSLOduration=9.610275353 podStartE2EDuration="28.415234367s" totalImagesPullingTime="18.804959014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:42.41381894 +0000 UTC m=+28.690560360" watchObservedRunningTime="2026-07-23 15:50:42.415234367 +0000 UTC m=+28.691975768" Jul 23 15:50:42.833336 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.833313 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tgdf5"] Jul 23 15:50:42.833440 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.833414 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:42.833503 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:42.833487 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:42.836233 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.836206 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gphkh"] Jul 23 15:50:42.836341 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.836286 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:42.836424 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:42.836365 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:42.848097 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.848073 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sg68n"] Jul 23 15:50:42.848188 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:42.848178 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:42.848297 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:42.848280 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:43.395043 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:43.395007 2547 generic.go:356] "Generic (PLEG): container finished" podID="f76a8e9b-f800-4ecc-ad18-08d9df0c8096" containerID="3eb6823719c713faafa9533c88ecf579bb947eba677c753cfc4d38c7c15a0b2c" exitCode=0 Jul 23 15:50:43.395433 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:43.395098 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerDied","Data":"3eb6823719c713faafa9533c88ecf579bb947eba677c753cfc4d38c7c15a0b2c"} Jul 23 15:50:44.235775 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:44.235606 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:44.235920 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:44.235665 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:44.235920 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:44.235851 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:44.235920 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:44.235905 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:45.234998 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:45.234976 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:45.235458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:45.235070 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:45.401984 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:45.401908 2547 generic.go:356] "Generic (PLEG): container finished" podID="f76a8e9b-f800-4ecc-ad18-08d9df0c8096" containerID="cff6c326bd8bafd1a60c1b4de824f849ac28e72aebcc5005e68adbbd81705721" exitCode=0 Jul 23 15:50:45.401984 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:45.401945 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerDied","Data":"cff6c326bd8bafd1a60c1b4de824f849ac28e72aebcc5005e68adbbd81705721"} Jul 23 15:50:46.234747 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:46.234712 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:46.234927 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:46.234826 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:46.234927 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:46.234876 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:46.235043 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:46.234958 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:47.235823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:47.235777 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:47.236278 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.235918 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sg68n" podUID="d199f478-aba8-452c-9224-ed736a726a29" Jul 23 15:50:47.846642 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:47.846562 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:47.846642 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:47.846628 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:47.846656 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846753 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846787 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846805 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846823 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846826 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret podName:b38c39bd-bd8b-40e9-95ad-c1ef17a463a0 nodeName:}" failed. No retries permitted until 2026-07-23 15:51:19.846806095 +0000 UTC m=+66.123547493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret") pod "global-pull-secret-syncer-gphkh" (UID: "b38c39bd-bd8b-40e9-95ad-c1ef17a463a0") : object "kube-system"/"original-pull-secret" not registered Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846835 2547 projected.go:196] Error preparing data for projected volume kube-api-access-mf69f for pod openshift-network-diagnostics/network-check-target-tgdf5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846846 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs podName:d199f478-aba8-452c-9224-ed736a726a29 nodeName:}" failed. No retries permitted until 2026-07-23 15:51:19.846836161 +0000 UTC m=+66.123577543 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs") pod "network-metrics-daemon-sg68n" (UID: "d199f478-aba8-452c-9224-ed736a726a29") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 15:50:47.846937 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:47.846879 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f podName:e15e83d8-73c2-47c3-95f3-6763d5f5114f nodeName:}" failed. No retries permitted until 2026-07-23 15:51:19.846862886 +0000 UTC m=+66.123604278 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mf69f" (UniqueName: "kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f") pod "network-check-target-tgdf5" (UID: "e15e83d8-73c2-47c3-95f3-6763d5f5114f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 15:50:48.235401 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:48.235314 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:48.235553 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:48.235442 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tgdf5" podUID="e15e83d8-73c2-47c3-95f3-6763d5f5114f" Jul 23 15:50:48.235553 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:48.235501 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:48.235654 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:48.235591 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gphkh" podUID="b38c39bd-bd8b-40e9-95ad-c1ef17a463a0" Jul 23 15:50:49.085482 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.085458 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-161.ec2.internal" event="NodeReady" Jul 23 15:50:49.086013 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.085604 2547 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 15:50:49.124064 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.124035 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b8977f69-dwpdp"] Jul 23 15:50:49.128491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.128465 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.131209 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.131185 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-69gss\"" Jul 23 15:50:49.131598 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.131547 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 15:50:49.131598 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.131560 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 15:50:49.131795 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.131678 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 15:50:49.141385 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.141054 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 15:50:49.143736 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.143715 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z"] Jul 23 15:50:49.147483 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.147462 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf"] Jul 23 15:50:49.147595 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.147576 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:49.150670 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.150650 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 23 15:50:49.150790 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.150707 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6qz66\"" Jul 23 15:50:49.150853 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.150840 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr"] Jul 23 15:50:49.150903 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.150874 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.150966 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.150949 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.151193 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.151176 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" Jul 23 15:50:49.153661 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.153642 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rnh5p\"" Jul 23 15:50:49.153913 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.153896 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8977f69-dwpdp"] Jul 23 15:50:49.154046 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.153925 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf"] Jul 23 15:50:49.154046 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.154032 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.154457 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.154440 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.154525 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.154495 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.156112 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.156094 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 23 15:50:49.156601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.156583 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wtfv7\"" Jul 23 15:50:49.156734 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.156720 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-nvzv6"] Jul 23 15:50:49.156912 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.156892 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.157709 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.157676 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.157805 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.157725 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.157805 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.157738 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 23 15:50:49.159394 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.159378 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 15:50:49.159484 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.159427 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.159731 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.159711 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sztxq"] Jul 23 15:50:49.159895 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.159878 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 15:50:49.159964 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.159910 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7lcx8\"" Jul 23 15:50:49.160051 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.160035 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.160169 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.160151 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.162730 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.162711 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4"] Jul 23 15:50:49.162933 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.162915 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.163600 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.163332 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 15:50:49.163600 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.163526 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.163769 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.163753 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 15:50:49.164172 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.164051 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.164577 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.164554 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-xt48d\"" Jul 23 15:50:49.166158 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.166142 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ds84x"] Jul 23 15:50:49.167131 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.166262 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.167720 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.167529 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.168916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.168274 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 15:50:49.168916 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.168605 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 15:50:49.172294 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.172277 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m"] Jul 23 15:50:49.173959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.172634 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.173959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.173249 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-njtxc\"" Jul 23 15:50:49.173959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.173257 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.173959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.173349 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 15:50:49.173959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.173376 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 15:50:49.173959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.173669 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hkhdd\"" Jul 23 15:50:49.173959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.173899 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.174430 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.174075 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.176493 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.176472 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-749854d9c6-jljn4"] Jul 23 15:50:49.179491 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.179283 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 15:50:49.180976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.180951 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l"] Jul 23 15:50:49.182781 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.182762 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.184218 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.184196 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:49.184991 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.184972 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c4nf5\"" Jul 23 15:50:49.185103 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.185090 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 15:50:49.188500 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.188480 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 15:50:49.189221 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.189198 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.189516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.189494 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tll2c\"" Jul 23 15:50:49.189717 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.189685 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 15:50:49.189919 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.189895 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 15:50:49.189995 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.189966 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 15:50:49.190171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.190149 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sbmkn"] Jul 23 15:50:49.190246 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.190239 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 15:50:49.190395 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.190380 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.190514 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.190498 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 15:50:49.190725 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.190704 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 15:50:49.190804 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.190769 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 15:50:49.191073 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.191054 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" Jul 23 15:50:49.191152 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.191079 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zlj67\"" Jul 23 15:50:49.195022 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195005 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.195117 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195087 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.195197 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195182 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z"] Jul 23 15:50:49.195255 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195209 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l"] Jul 23 15:50:49.195255 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195227 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf"] Jul 23 15:50:49.195255 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195243 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sztxq"] Jul 23 15:50:49.195397 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195260 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf"] Jul 23 15:50:49.195397 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195277 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4"] Jul 23 15:50:49.195397 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195296 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m"] Jul 23 15:50:49.195397 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195314 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-749854d9c6-jljn4"] Jul 23 15:50:49.195397 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195338 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ds84x"] Jul 23 15:50:49.195397 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195357 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sbmkn"] Jul 23 15:50:49.195678 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195409 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fdrkx\"" Jul 23 15:50:49.195678 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.195426 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.196726 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.196685 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-nvzv6"] Jul 23 15:50:49.197074 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.197037 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr"] Jul 23 15:50:49.197644 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.197624 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 15:50:49.198196 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.198178 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 15:50:49.198418 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.198186 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 15:50:49.198556 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.198540 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q8tlr\"" Jul 23 15:50:49.234866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.234849 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:50:49.238422 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.238137 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 15:50:49.238422 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.238393 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9k2x\"" Jul 23 15:50:49.257144 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257122 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw7dh\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-kube-api-access-pw7dh\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.257309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257172 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c75e171-26c0-455f-9380-f4a9d348fcda-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.257309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257196 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2c43bce-150e-49ca-b1e9-ff80e7521b67-config\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.257309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257225 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-bound-sa-token\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.257309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257241 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n94v5\" (UniqueName: \"kubernetes.io/projected/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-kube-api-access-n94v5\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.257309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257255 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2c43bce-150e-49ca-b1e9-ff80e7521b67-serving-cert\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.257309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257271 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.257309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257294 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98002b2a-d733-408d-9dd2-ae98f4273b6d-ca-trust-extracted\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257317 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-config\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257344 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257383 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92j77\" (UniqueName: \"kubernetes.io/projected/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-kube-api-access-92j77\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257415 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-installation-pull-secrets\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257440 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8766364-a437-4a90-9058-e1c2549638dc-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257465 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c75e171-26c0-455f-9380-f4a9d348fcda-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257522 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlz69\" (UniqueName: \"kubernetes.io/projected/6a3d81a0-7174-498a-be1d-c558568110ee-kube-api-access-mlz69\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257561 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8766364-a437-4a90-9058-e1c2549638dc-service-ca-bundle\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257595 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xhx9\" (UniqueName: \"kubernetes.io/projected/d2c43bce-150e-49ca-b1e9-ff80e7521b67-kube-api-access-5xhx9\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.257648 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257621 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-config-volume\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257653 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257730 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/561f3cb0-7a0d-4180-b795-504dee568a8e-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257766 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257791 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkl6p\" (UniqueName: \"kubernetes.io/projected/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-kube-api-access-rkl6p\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257840 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257873 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8766364-a437-4a90-9058-e1c2549638dc-tmp\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257896 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8766364-a437-4a90-9058-e1c2549638dc-snapshots\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257945 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-image-registry-private-configuration\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.257979 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-certificates\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258025 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2c43bce-150e-49ca-b1e9-ff80e7521b67-trusted-ca\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.258063 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258059 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hll8r\" (UniqueName: \"kubernetes.io/projected/a8766364-a437-4a90-9058-e1c2549638dc-kube-api-access-hll8r\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.258573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258114 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.258573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258143 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlcmq\" (UniqueName: \"kubernetes.io/projected/8b97bba7-ca5f-4c1b-b561-848da1d045e7-kube-api-access-xlcmq\") pod \"volume-data-source-validator-78cf4c6f6-hqksf\" (UID: \"8b97bba7-ca5f-4c1b-b561-848da1d045e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" Jul 23 15:50:49.258573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258205 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-tmp-dir\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.258573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258268 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-trusted-ca\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.258573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258298 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a3d81a0-7174-498a-be1d-c558568110ee-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.258573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258324 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8xs6\" (UniqueName: \"kubernetes.io/projected/9c75e171-26c0-455f-9380-f4a9d348fcda-kube-api-access-k8xs6\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.258573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.258353 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8766364-a437-4a90-9058-e1c2549638dc-serving-cert\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.359953 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.359868 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-config-volume\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.359953 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.359930 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.360142 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.359959 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4drqr\" (UniqueName: \"kubernetes.io/projected/72e133d7-09b8-4503-ac91-5a9b28006a5a-kube-api-access-4drqr\") pod \"network-check-source-7bc5b6bf85-fjt5l\" (UID: \"72e133d7-09b8-4503-ac91-5a9b28006a5a\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" Jul 23 15:50:49.360142 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.359995 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/561f3cb0-7a0d-4180-b795-504dee568a8e-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:49.360142 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360017 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.360142 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360122 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-stats-auth\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.360317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360160 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65e6104e-d595-4c17-9afd-685efbb4deca-cert\") pod \"ingress-canary-sbmkn\" (UID: \"65e6104e-d595-4c17-9afd-685efbb4deca\") " pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.360317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360198 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:49.360317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360227 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl6p\" (UniqueName: \"kubernetes.io/projected/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-kube-api-access-rkl6p\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:49.360317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360256 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.360317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360283 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8766364-a437-4a90-9058-e1c2549638dc-tmp\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.360317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360306 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8766364-a437-4a90-9058-e1c2549638dc-snapshots\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360330 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp4m5\" (UniqueName: \"kubernetes.io/projected/65e6104e-d595-4c17-9afd-685efbb4deca-kube-api-access-bp4m5\") pod \"ingress-canary-sbmkn\" (UID: \"65e6104e-d595-4c17-9afd-685efbb4deca\") " pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.360334 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360360 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-image-registry-private-configuration\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360370 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-config-volume\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.360393 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert podName:561f3cb0-7a0d-4180-b795-504dee568a8e nodeName:}" failed. No retries permitted until 2026-07-23 15:50:49.86037588 +0000 UTC m=+36.137117259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4sj5m" (UID: "561f3cb0-7a0d-4180-b795-504dee568a8e") : secret "networking-console-plugin-cert" not found Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360414 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-certificates\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360441 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2c43bce-150e-49ca-b1e9-ff80e7521b67-trusted-ca\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.360454 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360466 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hll8r\" (UniqueName: \"kubernetes.io/projected/a8766364-a437-4a90-9058-e1c2549638dc-kube-api-access-hll8r\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360492 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360515 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlcmq\" (UniqueName: \"kubernetes.io/projected/8b97bba7-ca5f-4c1b-b561-848da1d045e7-kube-api-access-xlcmq\") pod \"volume-data-source-validator-78cf4c6f6-hqksf\" (UID: \"8b97bba7-ca5f-4c1b-b561-848da1d045e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.360536 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls podName:6a3d81a0-7174-498a-be1d-c558568110ee nodeName:}" failed. No retries permitted until 2026-07-23 15:50:49.860519245 +0000 UTC m=+36.137260631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ctbdr" (UID: "6a3d81a0-7174-498a-be1d-c558568110ee") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:49.360603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360581 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-tmp-dir\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360638 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-trusted-ca\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360668 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a3d81a0-7174-498a-be1d-c558568110ee-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360718 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8xs6\" (UniqueName: \"kubernetes.io/projected/9c75e171-26c0-455f-9380-f4a9d348fcda-kube-api-access-k8xs6\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360743 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8766364-a437-4a90-9058-e1c2549638dc-serving-cert\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360757 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/561f3cb0-7a0d-4180-b795-504dee568a8e-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.360807 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw7dh\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-kube-api-access-pw7dh\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.361097 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a8766364-a437-4a90-9058-e1c2549638dc-snapshots\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.361269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.361268 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-tmp-dir\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.361662 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.361268 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-certificates\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.361662 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.361336 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8766364-a437-4a90-9058-e1c2549638dc-tmp\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.361790 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.361730 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2c43bce-150e-49ca-b1e9-ff80e7521b67-trusted-ca\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.361898 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.361880 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:50:49.361957 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.361928 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls podName:2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:49.861915597 +0000 UTC m=+36.138656976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls") pod "dns-default-ds84x" (UID: "2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6") : secret "dns-default-metrics-tls" not found Jul 23 15:50:49.362024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.361950 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a3d81a0-7174-498a-be1d-c558568110ee-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.362024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362007 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c75e171-26c0-455f-9380-f4a9d348fcda-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.362122 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362030 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2c43bce-150e-49ca-b1e9-ff80e7521b67-config\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.362122 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362053 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vps8g\" (UniqueName: \"kubernetes.io/projected/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-kube-api-access-vps8g\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.362122 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362079 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-bound-sa-token\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.362122 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362102 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n94v5\" (UniqueName: \"kubernetes.io/projected/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-kube-api-access-n94v5\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.362317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362166 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-trusted-ca\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.362317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362211 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2c43bce-150e-49ca-b1e9-ff80e7521b67-serving-cert\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.362317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362240 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-default-certificate\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.362317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362270 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.362317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362294 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98002b2a-d733-408d-9dd2-ae98f4273b6d-ca-trust-extracted\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362319 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-config\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362347 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362376 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92j77\" (UniqueName: \"kubernetes.io/projected/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-kube-api-access-92j77\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362401 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362431 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-installation-pull-secrets\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362457 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8766364-a437-4a90-9058-e1c2549638dc-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362488 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c75e171-26c0-455f-9380-f4a9d348fcda-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362516 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlz69\" (UniqueName: \"kubernetes.io/projected/6a3d81a0-7174-498a-be1d-c558568110ee-kube-api-access-mlz69\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.362554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362542 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8766364-a437-4a90-9058-e1c2549638dc-service-ca-bundle\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.362993 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362580 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c75e171-26c0-455f-9380-f4a9d348fcda-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.362993 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362580 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xhx9\" (UniqueName: \"kubernetes.io/projected/d2c43bce-150e-49ca-b1e9-ff80e7521b67-kube-api-access-5xhx9\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.362993 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.362613 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2c43bce-150e-49ca-b1e9-ff80e7521b67-config\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.362993 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.362742 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:50:49.362993 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.362898 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8977f69-dwpdp: secret "image-registry-tls" not found Jul 23 15:50:49.362993 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.362942 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls podName:98002b2a-d733-408d-9dd2-ae98f4273b6d nodeName:}" failed. No retries permitted until 2026-07-23 15:50:49.862928477 +0000 UTC m=+36.139669857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls") pod "image-registry-5b8977f69-dwpdp" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d") : secret "image-registry-tls" not found Jul 23 15:50:49.363274 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.363231 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98002b2a-d733-408d-9dd2-ae98f4273b6d-ca-trust-extracted\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.364723 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.363474 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8766364-a437-4a90-9058-e1c2549638dc-service-ca-bundle\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.364723 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.363792 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:50:49.364723 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.363859 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls podName:c85deb5a-7ad1-4904-ab1d-1878dd8e0599 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:49.863842619 +0000 UTC m=+36.140584002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tsr6z" (UID: "c85deb5a-7ad1-4904-ab1d-1878dd8e0599") : secret "samples-operator-tls" not found Jul 23 15:50:49.364723 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.364202 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-config\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.364723 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.364651 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8766364-a437-4a90-9058-e1c2549638dc-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.365364 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.365319 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8766364-a437-4a90-9058-e1c2549638dc-serving-cert\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.365364 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.365331 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-image-registry-private-configuration\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.365364 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.365332 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c75e171-26c0-455f-9380-f4a9d348fcda-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.365555 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.365413 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2c43bce-150e-49ca-b1e9-ff80e7521b67-serving-cert\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.365931 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.365903 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.367500 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.367478 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-installation-pull-secrets\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.379929 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.379873 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlcmq\" (UniqueName: \"kubernetes.io/projected/8b97bba7-ca5f-4c1b-b561-848da1d045e7-kube-api-access-xlcmq\") pod \"volume-data-source-validator-78cf4c6f6-hqksf\" (UID: \"8b97bba7-ca5f-4c1b-b561-848da1d045e7\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" Jul 23 15:50:49.380641 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.380477 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkl6p\" (UniqueName: \"kubernetes.io/projected/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-kube-api-access-rkl6p\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:49.380641 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.380599 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xhx9\" (UniqueName: \"kubernetes.io/projected/d2c43bce-150e-49ca-b1e9-ff80e7521b67-kube-api-access-5xhx9\") pod \"console-operator-565b7b569d-nvzv6\" (UID: \"d2c43bce-150e-49ca-b1e9-ff80e7521b67\") " pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.382493 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.381911 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-bound-sa-token\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.382493 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.381939 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n94v5\" (UniqueName: \"kubernetes.io/projected/6902bf59-cc55-492c-9dad-c3eb6c2ffa4c-kube-api-access-n94v5\") pod \"service-ca-operator-5bcbc94c78-9p8pf\" (UID: \"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.382493 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.382449 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlz69\" (UniqueName: \"kubernetes.io/projected/6a3d81a0-7174-498a-be1d-c558568110ee-kube-api-access-mlz69\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.382493 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.382461 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hll8r\" (UniqueName: \"kubernetes.io/projected/a8766364-a437-4a90-9058-e1c2549638dc-kube-api-access-hll8r\") pod \"insights-operator-555d9d7885-sztxq\" (UID: \"a8766364-a437-4a90-9058-e1c2549638dc\") " pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.382755 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.382537 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw7dh\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-kube-api-access-pw7dh\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.382866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.382841 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8xs6\" (UniqueName: \"kubernetes.io/projected/9c75e171-26c0-455f-9380-f4a9d348fcda-kube-api-access-k8xs6\") pod \"kube-storage-version-migrator-operator-7658d9b749-qsbp4\" (UID: \"9c75e171-26c0-455f-9380-f4a9d348fcda\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.383431 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.383395 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92j77\" (UniqueName: \"kubernetes.io/projected/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-kube-api-access-92j77\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.463967 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.463932 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vps8g\" (UniqueName: \"kubernetes.io/projected/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-kube-api-access-vps8g\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.464145 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.463977 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-default-certificate\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.464145 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.464038 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.464145 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.464078 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4drqr\" (UniqueName: \"kubernetes.io/projected/72e133d7-09b8-4503-ac91-5a9b28006a5a-kube-api-access-4drqr\") pod \"network-check-source-7bc5b6bf85-fjt5l\" (UID: \"72e133d7-09b8-4503-ac91-5a9b28006a5a\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" Jul 23 15:50:49.464308 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.464181 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:50:49.464308 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.464205 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.464308 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.464245 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:49.964225995 +0000 UTC m=+36.240967373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : secret "router-metrics-certs-default" not found Jul 23 15:50:49.464308 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.464264 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-stats-auth\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.464308 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.464290 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65e6104e-d595-4c17-9afd-685efbb4deca-cert\") pod \"ingress-canary-sbmkn\" (UID: \"65e6104e-d595-4c17-9afd-685efbb4deca\") " pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.464308 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.464311 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:49.964287396 +0000 UTC m=+36.241028793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : configmap references non-existent config key: service-ca.crt Jul 23 15:50:49.464596 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.464382 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp4m5\" (UniqueName: \"kubernetes.io/projected/65e6104e-d595-4c17-9afd-685efbb4deca-kube-api-access-bp4m5\") pod \"ingress-canary-sbmkn\" (UID: \"65e6104e-d595-4c17-9afd-685efbb4deca\") " pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.466831 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.466801 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-default-certificate\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.466964 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.466948 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-stats-auth\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.467023 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.466960 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65e6104e-d595-4c17-9afd-685efbb4deca-cert\") pod \"ingress-canary-sbmkn\" (UID: \"65e6104e-d595-4c17-9afd-685efbb4deca\") " pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.468619 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.468588 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" Jul 23 15:50:49.472786 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.472768 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp4m5\" (UniqueName: \"kubernetes.io/projected/65e6104e-d595-4c17-9afd-685efbb4deca-kube-api-access-bp4m5\") pod \"ingress-canary-sbmkn\" (UID: \"65e6104e-d595-4c17-9afd-685efbb4deca\") " pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.473683 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.473660 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vps8g\" (UniqueName: \"kubernetes.io/projected/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-kube-api-access-vps8g\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.474145 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.474078 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4drqr\" (UniqueName: \"kubernetes.io/projected/72e133d7-09b8-4503-ac91-5a9b28006a5a-kube-api-access-4drqr\") pod \"network-check-source-7bc5b6bf85-fjt5l\" (UID: \"72e133d7-09b8-4503-ac91-5a9b28006a5a\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" Jul 23 15:50:49.491467 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.491430 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" Jul 23 15:50:49.502460 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.502437 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:49.509132 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.509116 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-sztxq" Jul 23 15:50:49.514893 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.514871 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" Jul 23 15:50:49.542046 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.542013 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" Jul 23 15:50:49.548009 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.547983 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sbmkn" Jul 23 15:50:49.868427 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.868395 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:49.868427 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.868432 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.868457 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.868524 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.868556 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868568 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868609 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868568 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868643 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868663 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8977f69-dwpdp: secret "image-registry-tls" not found Jul 23 15:50:49.868677 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868645 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:50:49.869102 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868642 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls podName:6a3d81a0-7174-498a-be1d-c558568110ee nodeName:}" failed. No retries permitted until 2026-07-23 15:50:50.868622349 +0000 UTC m=+37.145363729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ctbdr" (UID: "6a3d81a0-7174-498a-be1d-c558568110ee") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:49.869102 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868760 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls podName:2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:50.868742674 +0000 UTC m=+37.145484056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls") pod "dns-default-ds84x" (UID: "2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6") : secret "dns-default-metrics-tls" not found Jul 23 15:50:49.869102 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868850 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert podName:561f3cb0-7a0d-4180-b795-504dee568a8e nodeName:}" failed. No retries permitted until 2026-07-23 15:50:50.868836883 +0000 UTC m=+37.145578266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4sj5m" (UID: "561f3cb0-7a0d-4180-b795-504dee568a8e") : secret "networking-console-plugin-cert" not found Jul 23 15:50:49.869102 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868880 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls podName:98002b2a-d733-408d-9dd2-ae98f4273b6d nodeName:}" failed. No retries permitted until 2026-07-23 15:50:50.868869012 +0000 UTC m=+37.145610391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls") pod "image-registry-5b8977f69-dwpdp" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d") : secret "image-registry-tls" not found Jul 23 15:50:49.869102 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.868898 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls podName:c85deb5a-7ad1-4904-ab1d-1878dd8e0599 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:50.868889774 +0000 UTC m=+37.145631160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tsr6z" (UID: "c85deb5a-7ad1-4904-ab1d-1878dd8e0599") : secret "samples-operator-tls" not found Jul 23 15:50:49.969759 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.969723 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.969950 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.969928 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:50.969908641 +0000 UTC m=+37.246650029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : configmap references non-existent config key: service-ca.crt Jul 23 15:50:49.970054 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:49.970039 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:49.970195 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.970173 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:50:49.970283 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:49.970242 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:50.970223831 +0000 UTC m=+37.246965213 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : secret "router-metrics-certs-default" not found Jul 23 15:50:50.235985 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.235391 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:50:50.238546 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.236940 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:50:50.242176 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.240758 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xglcr\"" Jul 23 15:50:50.242176 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.241044 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 15:50:50.459262 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.458950 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf"] Jul 23 15:50:50.464234 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:50.464207 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b97bba7_ca5f_4c1b_b561_848da1d045e7.slice/crio-b431cfe9ec303f5342565b62537bfbdfcb3c5559431e7ec1097e189a71ef98f0 WatchSource:0}: Error finding container b431cfe9ec303f5342565b62537bfbdfcb3c5559431e7ec1097e189a71ef98f0: Status 404 returned error can't find the container with id b431cfe9ec303f5342565b62537bfbdfcb3c5559431e7ec1097e189a71ef98f0 Jul 23 15:50:50.471097 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.471073 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4"] Jul 23 15:50:50.475970 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.475950 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sbmkn"] Jul 23 15:50:50.479001 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:50.476821 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c75e171_26c0_455f_9380_f4a9d348fcda.slice/crio-afd9e9015a55a2da832cf511288110f0d7be68bcc1becf4c6d12c23f02409b8a WatchSource:0}: Error finding container afd9e9015a55a2da832cf511288110f0d7be68bcc1becf4c6d12c23f02409b8a: Status 404 returned error can't find the container with id afd9e9015a55a2da832cf511288110f0d7be68bcc1becf4c6d12c23f02409b8a Jul 23 15:50:50.479001 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.478363 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-nvzv6"] Jul 23 15:50:50.483518 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:50.482939 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c43bce_150e_49ca_b1e9_ff80e7521b67.slice/crio-92f203a5964ff4ae71bf93a25c0616f20aabd1a4a57e1414e17a87392fb4f212 WatchSource:0}: Error finding container 92f203a5964ff4ae71bf93a25c0616f20aabd1a4a57e1414e17a87392fb4f212: Status 404 returned error can't find the container with id 92f203a5964ff4ae71bf93a25c0616f20aabd1a4a57e1414e17a87392fb4f212 Jul 23 15:50:50.498883 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.498363 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l"] Jul 23 15:50:50.499620 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.499602 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-sztxq"] Jul 23 15:50:50.503553 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:50.503526 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e133d7_09b8_4503_ac91_5a9b28006a5a.slice/crio-43ee97ebf241f87074c2386b09603875d7ad9b870b2b08370bc30413c390016a WatchSource:0}: Error finding container 43ee97ebf241f87074c2386b09603875d7ad9b870b2b08370bc30413c390016a: Status 404 returned error can't find the container with id 43ee97ebf241f87074c2386b09603875d7ad9b870b2b08370bc30413c390016a Jul 23 15:50:50.505664 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:50.505641 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8766364_a437_4a90_9058_e1c2549638dc.slice/crio-d1c54adac4c0b11101e6fd4a6e57f43361e6d983440f260efd838360d131d01a WatchSource:0}: Error finding container d1c54adac4c0b11101e6fd4a6e57f43361e6d983440f260efd838360d131d01a: Status 404 returned error can't find the container with id d1c54adac4c0b11101e6fd4a6e57f43361e6d983440f260efd838360d131d01a Jul 23 15:50:50.512250 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.512227 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf"] Jul 23 15:50:50.515028 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:50:50.515009 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6902bf59_cc55_492c_9dad_c3eb6c2ffa4c.slice/crio-262024257eab04de0de4d23e2d92325e3c2f0622b4fc26dc1f97d0a70d6e042b WatchSource:0}: Error finding container 262024257eab04de0de4d23e2d92325e3c2f0622b4fc26dc1f97d0a70d6e042b: Status 404 returned error can't find the container with id 262024257eab04de0de4d23e2d92325e3c2f0622b4fc26dc1f97d0a70d6e042b Jul 23 15:50:50.879486 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.879450 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:50.879486 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.879487 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:50.879762 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.879511 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:50.879762 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.879556 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:50.879762 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.879576 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:50.879762 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.879625 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:50.879984 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.879966 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls podName:6a3d81a0-7174-498a-be1d-c558568110ee nodeName:}" failed. No retries permitted until 2026-07-23 15:50:52.879939257 +0000 UTC m=+39.156680640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ctbdr" (UID: "6a3d81a0-7174-498a-be1d-c558568110ee") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:50.880066 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880014 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:50:50.880116 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880064 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:50:50.880116 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880079 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:50:50.880116 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880086 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8977f69-dwpdp: secret "image-registry-tls" not found Jul 23 15:50:50.880116 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880105 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls podName:c85deb5a-7ad1-4904-ab1d-1878dd8e0599 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:52.880078043 +0000 UTC m=+39.156819437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tsr6z" (UID: "c85deb5a-7ad1-4904-ab1d-1878dd8e0599") : secret "samples-operator-tls" not found Jul 23 15:50:50.880116 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880022 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:50:50.880411 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880148 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls podName:2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:52.880129072 +0000 UTC m=+39.156870455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls") pod "dns-default-ds84x" (UID: "2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6") : secret "dns-default-metrics-tls" not found Jul 23 15:50:50.880411 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880177 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert podName:561f3cb0-7a0d-4180-b795-504dee568a8e nodeName:}" failed. No retries permitted until 2026-07-23 15:50:52.880157901 +0000 UTC m=+39.156899281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4sj5m" (UID: "561f3cb0-7a0d-4180-b795-504dee568a8e") : secret "networking-console-plugin-cert" not found Jul 23 15:50:50.880411 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.880195 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls podName:98002b2a-d733-408d-9dd2-ae98f4273b6d nodeName:}" failed. No retries permitted until 2026-07-23 15:50:52.88018432 +0000 UTC m=+39.156925699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls") pod "image-registry-5b8977f69-dwpdp" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d") : secret "image-registry-tls" not found Jul 23 15:50:50.980468 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.980423 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:50.980631 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:50.980487 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:50.980631 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.980562 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:50:50.980727 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.980645 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:52.980624144 +0000 UTC m=+39.257365536 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : secret "router-metrics-certs-default" not found Jul 23 15:50:50.980727 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:50.980665 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:52.980654955 +0000 UTC m=+39.257396338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : configmap references non-existent config key: service-ca.crt Jul 23 15:50:51.430312 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.430101 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" event={"ID":"72e133d7-09b8-4503-ac91-5a9b28006a5a","Type":"ContainerStarted","Data":"43ee97ebf241f87074c2386b09603875d7ad9b870b2b08370bc30413c390016a"} Jul 23 15:50:51.433667 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.433615 2547 generic.go:356] "Generic (PLEG): container finished" podID="f76a8e9b-f800-4ecc-ad18-08d9df0c8096" containerID="5ab250ec555dee63cc143c1ed4cb81309a1609e40900d8eaf930701f2db11c7e" exitCode=0 Jul 23 15:50:51.433811 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.433711 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerDied","Data":"5ab250ec555dee63cc143c1ed4cb81309a1609e40900d8eaf930701f2db11c7e"} Jul 23 15:50:51.435247 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.435218 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" event={"ID":"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c","Type":"ContainerStarted","Data":"262024257eab04de0de4d23e2d92325e3c2f0622b4fc26dc1f97d0a70d6e042b"} Jul 23 15:50:51.437056 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.437026 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sztxq" event={"ID":"a8766364-a437-4a90-9058-e1c2549638dc","Type":"ContainerStarted","Data":"d1c54adac4c0b11101e6fd4a6e57f43361e6d983440f260efd838360d131d01a"} Jul 23 15:50:51.438286 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.438253 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" event={"ID":"d2c43bce-150e-49ca-b1e9-ff80e7521b67","Type":"ContainerStarted","Data":"92f203a5964ff4ae71bf93a25c0616f20aabd1a4a57e1414e17a87392fb4f212"} Jul 23 15:50:51.439886 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.439863 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" event={"ID":"8b97bba7-ca5f-4c1b-b561-848da1d045e7","Type":"ContainerStarted","Data":"b431cfe9ec303f5342565b62537bfbdfcb3c5559431e7ec1097e189a71ef98f0"} Jul 23 15:50:51.441025 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.441004 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sbmkn" event={"ID":"65e6104e-d595-4c17-9afd-685efbb4deca","Type":"ContainerStarted","Data":"7b63bfdb6f1e3790d0052f6ea1f2cdd9ae21f52f11b9e18e8581e9d1f59c562d"} Jul 23 15:50:51.442185 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:51.442139 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" event={"ID":"9c75e171-26c0-455f-9380-f4a9d348fcda","Type":"ContainerStarted","Data":"afd9e9015a55a2da832cf511288110f0d7be68bcc1becf4c6d12c23f02409b8a"} Jul 23 15:50:52.454480 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:52.453437 2547 generic.go:356] "Generic (PLEG): container finished" podID="f76a8e9b-f800-4ecc-ad18-08d9df0c8096" containerID="5ca4fcaad5b648f79eff650e68e8a925aeb1d9e330d3f284c2f90d4fa4975c8b" exitCode=0 Jul 23 15:50:52.454480 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:52.453518 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerDied","Data":"5ca4fcaad5b648f79eff650e68e8a925aeb1d9e330d3f284c2f90d4fa4975c8b"} Jul 23 15:50:52.899300 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:52.899261 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:52.899457 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:52.899313 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:52.899457 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:52.899346 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:52.899457 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:52.899416 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:52.899457 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:52.899442 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:52.899681 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.899663 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.899790 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.899867 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls podName:6a3d81a0-7174-498a-be1d-c558568110ee nodeName:}" failed. No retries permitted until 2026-07-23 15:50:56.899847655 +0000 UTC m=+43.176589055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ctbdr" (UID: "6a3d81a0-7174-498a-be1d-c558568110ee") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.900264 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.900313 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert podName:561f3cb0-7a0d-4180-b795-504dee568a8e nodeName:}" failed. No retries permitted until 2026-07-23 15:50:56.900297782 +0000 UTC m=+43.177039178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4sj5m" (UID: "561f3cb0-7a0d-4180-b795-504dee568a8e") : secret "networking-console-plugin-cert" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.900378 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.900408 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls podName:2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:56.900397727 +0000 UTC m=+43.177139115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls") pod "dns-default-ds84x" (UID: "2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6") : secret "dns-default-metrics-tls" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.900472 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.900482 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8977f69-dwpdp: secret "image-registry-tls" not found Jul 23 15:50:52.900642 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.900511 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls podName:98002b2a-d733-408d-9dd2-ae98f4273b6d nodeName:}" failed. No retries permitted until 2026-07-23 15:50:56.900501221 +0000 UTC m=+43.177242616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls") pod "image-registry-5b8977f69-dwpdp" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d") : secret "image-registry-tls" not found Jul 23 15:50:52.901364 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:52.901297 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls podName:c85deb5a-7ad1-4904-ab1d-1878dd8e0599 nodeName:}" failed. No retries permitted until 2026-07-23 15:50:56.90127536 +0000 UTC m=+43.178016754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tsr6z" (UID: "c85deb5a-7ad1-4904-ab1d-1878dd8e0599") : secret "samples-operator-tls" not found Jul 23 15:50:53.001668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:53.000765 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:53.001668 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:53.000957 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:53.001668 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:53.001085 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:50:53.001668 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:53.001146 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:57.001127694 +0000 UTC m=+43.277869094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : secret "router-metrics-certs-default" not found Jul 23 15:50:53.001668 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:53.001542 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:50:57.001521854 +0000 UTC m=+43.278263260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : configmap references non-existent config key: service-ca.crt Jul 23 15:50:56.940795 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:56.940749 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:56.940813 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:56.940923 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.940931 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.940952 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8977f69-dwpdp: secret "image-registry-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:56.940955 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:56.940994 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941018 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls podName:98002b2a-d733-408d-9dd2-ae98f4273b6d nodeName:}" failed. No retries permitted until 2026-07-23 15:51:04.940995378 +0000 UTC m=+51.217736771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls") pod "image-registry-5b8977f69-dwpdp" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d") : secret "image-registry-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941125 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941165 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941174 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls podName:c85deb5a-7ad1-4904-ab1d-1878dd8e0599 nodeName:}" failed. No retries permitted until 2026-07-23 15:51:04.941158901 +0000 UTC m=+51.217900284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tsr6z" (UID: "c85deb5a-7ad1-4904-ab1d-1878dd8e0599") : secret "samples-operator-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941205 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls podName:2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6 nodeName:}" failed. No retries permitted until 2026-07-23 15:51:04.941193139 +0000 UTC m=+51.217934535 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls") pod "dns-default-ds84x" (UID: "2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6") : secret "dns-default-metrics-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941334 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941367 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert podName:561f3cb0-7a0d-4180-b795-504dee568a8e nodeName:}" failed. No retries permitted until 2026-07-23 15:51:04.941356042 +0000 UTC m=+51.218097459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4sj5m" (UID: "561f3cb0-7a0d-4180-b795-504dee568a8e") : secret "networking-console-plugin-cert" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941437 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:56.941458 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:56.941468 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls podName:6a3d81a0-7174-498a-be1d-c558568110ee nodeName:}" failed. No retries permitted until 2026-07-23 15:51:04.941457958 +0000 UTC m=+51.218199339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ctbdr" (UID: "6a3d81a0-7174-498a-be1d-c558568110ee") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:50:57.041787 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:57.041748 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:57.041926 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:57.041861 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:50:57.041926 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:57.041906 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:51:05.041883742 +0000 UTC m=+51.318625120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : configmap references non-existent config key: service-ca.crt Jul 23 15:50:57.042013 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:57.041958 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:50:57.042013 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:50:57.041999 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:51:05.041988983 +0000 UTC m=+51.318730362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : secret "router-metrics-certs-default" not found Jul 23 15:50:59.480909 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.480869 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" event={"ID":"f76a8e9b-f800-4ecc-ad18-08d9df0c8096","Type":"ContainerStarted","Data":"2c0b01df1b8a1a6c62831bd23d0c0eb1341f66d8563cf48bebfd3a4a6a25695b"} Jul 23 15:50:59.483069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.483024 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" event={"ID":"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c","Type":"ContainerStarted","Data":"f4e45c3c7d7461c26aba76834d9c886f7122c019c7849b9784997fd97ac65fb8"} Jul 23 15:50:59.485224 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.485173 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sztxq" event={"ID":"a8766364-a437-4a90-9058-e1c2549638dc","Type":"ContainerStarted","Data":"7acd7aeb499c704ae4649d6c58274af350a615506fc9a3b3fb30ae3d35b328a5"} Jul 23 15:50:59.488272 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.488252 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/0.log" Jul 23 15:50:59.488377 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.488291 2547 generic.go:356] "Generic (PLEG): container finished" podID="d2c43bce-150e-49ca-b1e9-ff80e7521b67" containerID="03bf356feea74da4f8c9c5289c4e54ef84332028d50f7a6e81090687540775d6" exitCode=255 Jul 23 15:50:59.488377 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.488363 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" event={"ID":"d2c43bce-150e-49ca-b1e9-ff80e7521b67","Type":"ContainerDied","Data":"03bf356feea74da4f8c9c5289c4e54ef84332028d50f7a6e81090687540775d6"} Jul 23 15:50:59.488535 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.488519 2547 scope.go:117] "RemoveContainer" containerID="03bf356feea74da4f8c9c5289c4e54ef84332028d50f7a6e81090687540775d6" Jul 23 15:50:59.490265 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.490239 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" event={"ID":"8b97bba7-ca5f-4c1b-b561-848da1d045e7","Type":"ContainerStarted","Data":"6fd1c3127e30c47639a81bf7e6da7a7752b25a458df1f2bf5daee97ac995c8d3"} Jul 23 15:50:59.492029 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.491813 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sbmkn" event={"ID":"65e6104e-d595-4c17-9afd-685efbb4deca","Type":"ContainerStarted","Data":"e5bf09a45bb8d1d2615e9e3c29b1fa4819acb1a1d45be5ad2b3d72655f327792"} Jul 23 15:50:59.493569 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.493548 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" event={"ID":"9c75e171-26c0-455f-9380-f4a9d348fcda","Type":"ContainerStarted","Data":"5d9e324e4bdd6cedc1e5c3a83d77c67698c750e0e52c60912f1746c4656eba51"} Jul 23 15:50:59.495417 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.495397 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" event={"ID":"72e133d7-09b8-4503-ac91-5a9b28006a5a","Type":"ContainerStarted","Data":"7b3f4beb65f61957a1967ef7317a613d51ec9a1e8f7e3a9052373e86cbf21fb6"} Jul 23 15:50:59.503467 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.503447 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:59.503467 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.503486 2547 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:50:59.505897 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.505831 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ql7xx" podStartSLOduration=14.541865922 podStartE2EDuration="45.505814205s" totalImagesPullingTime="30.963948283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:59.50388572 +0000 UTC m=+45.780627114" watchObservedRunningTime="2026-07-23 15:50:59.505814205 +0000 UTC m=+45.782555607" Jul 23 15:50:59.536891 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.536843 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" podStartSLOduration=35.456894021 podStartE2EDuration="43.536821046s" totalImagesPullingTime="8.079927025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:59.535843224 +0000 UTC m=+45.812584626" watchObservedRunningTime="2026-07-23 15:50:59.536821046 +0000 UTC m=+45.813562428" Jul 23 15:50:59.557613 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.557565 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-sztxq" podStartSLOduration=35.503845754 podStartE2EDuration="43.557545066s" totalImagesPullingTime="8.053699312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:59.556403469 +0000 UTC m=+45.833144872" watchObservedRunningTime="2026-07-23 15:50:59.557545066 +0000 UTC m=+45.834286469" Jul 23 15:50:59.576948 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.576899 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-hqksf" podStartSLOduration=35.481853185 podStartE2EDuration="43.57687249s" totalImagesPullingTime="8.095019305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:59.571318752 +0000 UTC m=+45.848060154" watchObservedRunningTime="2026-07-23 15:50:59.57687249 +0000 UTC m=+45.853613892" Jul 23 15:50:59.590254 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.590192 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" podStartSLOduration=35.546210937 podStartE2EDuration="43.590173916s" totalImagesPullingTime="8.043962979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:59.589222549 +0000 UTC m=+45.865963948" watchObservedRunningTime="2026-07-23 15:50:59.590173916 +0000 UTC m=+45.866915318" Jul 23 15:50:59.607059 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.605935 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-fjt5l" podStartSLOduration=35.549147179 podStartE2EDuration="43.60591758s" totalImagesPullingTime="8.056770401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:59.605092762 +0000 UTC m=+45.881834163" watchObservedRunningTime="2026-07-23 15:50:59.60591758 +0000 UTC m=+45.882658983" Jul 23 15:50:59.949301 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.949241 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sbmkn" podStartSLOduration=2.8625226980000003 podStartE2EDuration="10.949214467s" totalImagesPullingTime="8.086691769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:50:59.62858835 +0000 UTC m=+45.905329752" watchObservedRunningTime="2026-07-23 15:50:59.949214467 +0000 UTC m=+46.225955869" Jul 23 15:50:59.950250 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.950229 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-txdfw"] Jul 23 15:50:59.963653 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.963251 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:50:59.965195 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.965169 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-txdfw"] Jul 23 15:50:59.966302 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.966279 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 15:50:59.966756 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.966737 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n8xfp\"" Jul 23 15:50:59.967031 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:50:59.966743 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 15:51:00.070831 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.070793 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6170b78e-67ba-4e41-9ff7-95b1602719da-crio-socket\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.070979 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.070888 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6170b78e-67ba-4e41-9ff7-95b1602719da-data-volume\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.070979 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.070959 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.071044 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.071034 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.071088 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.071074 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xkdh\" (UniqueName: \"kubernetes.io/projected/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-api-access-7xkdh\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.172239 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.172203 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6170b78e-67ba-4e41-9ff7-95b1602719da-crio-socket\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.172393 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.172272 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6170b78e-67ba-4e41-9ff7-95b1602719da-data-volume\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.172393 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.172320 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.172393 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.172386 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.172505 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.172402 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6170b78e-67ba-4e41-9ff7-95b1602719da-crio-socket\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.172505 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.172425 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xkdh\" (UniqueName: \"kubernetes.io/projected/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-api-access-7xkdh\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.172505 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.172481 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:51:00.172594 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.172512 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:51:00.172594 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.172547 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:00.672531271 +0000 UTC m=+46.949272653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : configmap "kube-rbac-proxy" not found Jul 23 15:51:00.172594 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.172569 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:00.672553863 +0000 UTC m=+46.949295242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : secret "insights-runtime-extractor-tls" not found Jul 23 15:51:00.172727 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.172657 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6170b78e-67ba-4e41-9ff7-95b1602719da-data-volume\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.184637 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.184611 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xkdh\" (UniqueName: \"kubernetes.io/projected/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-api-access-7xkdh\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.500573 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.500545 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 15:51:00.500958 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.500941 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/0.log" Jul 23 15:51:00.501002 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.500976 2547 generic.go:356] "Generic (PLEG): container finished" podID="d2c43bce-150e-49ca-b1e9-ff80e7521b67" containerID="1ba55ec84caf06f0881505981729c8e48030f9585c9ca9e040300f14a9675f69" exitCode=255 Jul 23 15:51:00.501041 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.501008 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" event={"ID":"d2c43bce-150e-49ca-b1e9-ff80e7521b67","Type":"ContainerDied","Data":"1ba55ec84caf06f0881505981729c8e48030f9585c9ca9e040300f14a9675f69"} Jul 23 15:51:00.501090 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.501075 2547 scope.go:117] "RemoveContainer" containerID="03bf356feea74da4f8c9c5289c4e54ef84332028d50f7a6e81090687540775d6" Jul 23 15:51:00.501317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.501288 2547 scope.go:117] "RemoveContainer" containerID="1ba55ec84caf06f0881505981729c8e48030f9585c9ca9e040300f14a9675f69" Jul 23 15:51:00.501483 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.501449 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-nvzv6_openshift-console-operator(d2c43bce-150e-49ca-b1e9-ff80e7521b67)\"" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" podUID="d2c43bce-150e-49ca-b1e9-ff80e7521b67" Jul 23 15:51:00.676790 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.676752 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.676979 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:00.676820 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:00.676979 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.676892 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:51:00.676979 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.676926 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:51:00.676979 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.676948 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:01.676934967 +0000 UTC m=+47.953676347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : configmap "kube-rbac-proxy" not found Jul 23 15:51:00.677132 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:00.677066 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:01.677046296 +0000 UTC m=+47.953787687 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : secret "insights-runtime-extractor-tls" not found Jul 23 15:51:01.506424 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:01.506396 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 15:51:01.506809 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:01.506729 2547 scope.go:117] "RemoveContainer" containerID="1ba55ec84caf06f0881505981729c8e48030f9585c9ca9e040300f14a9675f69" Jul 23 15:51:01.506901 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:01.506882 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-nvzv6_openshift-console-operator(d2c43bce-150e-49ca-b1e9-ff80e7521b67)\"" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" podUID="d2c43bce-150e-49ca-b1e9-ff80e7521b67" Jul 23 15:51:01.686584 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:01.686544 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:01.686776 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:01.686675 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:51:01.686776 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:01.686745 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:01.686776 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:01.686756 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:03.686739074 +0000 UTC m=+49.963480453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : configmap "kube-rbac-proxy" not found Jul 23 15:51:01.686971 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:01.686880 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:51:01.686971 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:01.686953 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:03.686937004 +0000 UTC m=+49.963678388 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : secret "insights-runtime-extractor-tls" not found Jul 23 15:51:02.153432 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:02.153405 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rrxrl_e7d9a113-8cc2-42e8-807c-58db20ab3221/dns-node-resolver/0.log" Jul 23 15:51:02.754217 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:02.754189 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lx88p_029afce9-440f-4b8c-992f-97e2786871bf/node-ca/0.log" Jul 23 15:51:03.704713 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:03.704648 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:03.704884 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:03.704777 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 15:51:03.704884 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:03.704806 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:03.704884 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:03.704849 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:07.704833575 +0000 UTC m=+53.981574955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : configmap "kube-rbac-proxy" not found Jul 23 15:51:03.705016 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:03.704900 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:51:03.705016 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:03.704952 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:07.704941429 +0000 UTC m=+53.981682808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : secret "insights-runtime-extractor-tls" not found Jul 23 15:51:03.754355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:03.754327 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sbmkn_65e6104e-d595-4c17-9afd-685efbb4deca/serve-healthcheck-canary/0.log" Jul 23 15:51:05.017376 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:05.017336 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:51:05.017376 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:05.017382 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:05.017408 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:05.017460 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:05.017488 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017498 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017547 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017580 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert podName:561f3cb0-7a0d-4180-b795-504dee568a8e nodeName:}" failed. No retries permitted until 2026-07-23 15:51:21.017560403 +0000 UTC m=+67.294301807 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4sj5m" (UID: "561f3cb0-7a0d-4180-b795-504dee568a8e") : secret "networking-console-plugin-cert" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017601 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017610 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls podName:6a3d81a0-7174-498a-be1d-c558568110ee nodeName:}" failed. No retries permitted until 2026-07-23 15:51:21.017593563 +0000 UTC m=+67.294334960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ctbdr" (UID: "6a3d81a0-7174-498a-be1d-c558568110ee") : secret "cluster-monitoring-operator-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017555 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017644 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls podName:c85deb5a-7ad1-4904-ab1d-1878dd8e0599 nodeName:}" failed. No retries permitted until 2026-07-23 15:51:21.017633027 +0000 UTC m=+67.294374435 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tsr6z" (UID: "c85deb5a-7ad1-4904-ab1d-1878dd8e0599") : secret "samples-operator-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017601 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017717 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8977f69-dwpdp: secret "image-registry-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017721 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls podName:2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6 nodeName:}" failed. No retries permitted until 2026-07-23 15:51:21.017679791 +0000 UTC m=+67.294421177 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls") pod "dns-default-ds84x" (UID: "2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6") : secret "dns-default-metrics-tls" not found Jul 23 15:51:05.017954 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.017781 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls podName:98002b2a-d733-408d-9dd2-ae98f4273b6d nodeName:}" failed. No retries permitted until 2026-07-23 15:51:21.017770809 +0000 UTC m=+67.294512191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls") pod "image-registry-5b8977f69-dwpdp" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d") : secret "image-registry-tls" not found Jul 23 15:51:05.118394 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:05.118354 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:05.118568 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.118515 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:51:21.118500099 +0000 UTC m=+67.395241481 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : configmap references non-existent config key: service-ca.crt Jul 23 15:51:05.118568 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:05.118542 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:05.118664 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.118636 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 15:51:05.118727 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:05.118666 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs podName:baff5aa4-e1cc-4e0c-baba-6f2711cdaacf nodeName:}" failed. No retries permitted until 2026-07-23 15:51:21.118655322 +0000 UTC m=+67.395396708 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs") pod "router-default-749854d9c6-jljn4" (UID: "baff5aa4-e1cc-4e0c-baba-6f2711cdaacf") : secret "router-metrics-certs-default" not found Jul 23 15:51:07.739365 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:07.739332 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:07.739909 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:07.739456 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:07.739909 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:07.739544 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 15:51:07.739909 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:07.739595 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls podName:6170b78e-67ba-4e41-9ff7-95b1602719da nodeName:}" failed. No retries permitted until 2026-07-23 15:51:15.739581456 +0000 UTC m=+62.016322834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-txdfw" (UID: "6170b78e-67ba-4e41-9ff7-95b1602719da") : secret "insights-runtime-extractor-tls" not found Jul 23 15:51:07.752388 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:07.752359 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6170b78e-67ba-4e41-9ff7-95b1602719da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:09.503297 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:09.503256 2547 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:51:09.503781 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:09.503320 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:51:09.503863 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:09.503844 2547 scope.go:117] "RemoveContainer" containerID="1ba55ec84caf06f0881505981729c8e48030f9585c9ca9e040300f14a9675f69" Jul 23 15:51:09.504040 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:09.504015 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-nvzv6_openshift-console-operator(d2c43bce-150e-49ca-b1e9-ff80e7521b67)\"" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" podUID="d2c43bce-150e-49ca-b1e9-ff80e7521b67" Jul 23 15:51:13.405866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:13.405837 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4qjk" Jul 23 15:51:15.814332 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:15.814294 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:15.816544 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:15.816524 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6170b78e-67ba-4e41-9ff7-95b1602719da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-txdfw\" (UID: \"6170b78e-67ba-4e41-9ff7-95b1602719da\") " pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:15.876774 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:15.876746 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n8xfp\"" Jul 23 15:51:15.884040 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:15.884022 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-txdfw" Jul 23 15:51:16.008474 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:16.008438 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-txdfw"] Jul 23 15:51:16.010922 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:16.010898 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6170b78e_67ba_4e41_9ff7_95b1602719da.slice/crio-c7e30e52efd6a5a2527c8630886df125a854a1fc4b73a8cdb7860049334f519a WatchSource:0}: Error finding container c7e30e52efd6a5a2527c8630886df125a854a1fc4b73a8cdb7860049334f519a: Status 404 returned error can't find the container with id c7e30e52efd6a5a2527c8630886df125a854a1fc4b73a8cdb7860049334f519a Jul 23 15:51:16.557201 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:16.557159 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txdfw" event={"ID":"6170b78e-67ba-4e41-9ff7-95b1602719da","Type":"ContainerStarted","Data":"c7e30e52efd6a5a2527c8630886df125a854a1fc4b73a8cdb7860049334f519a"} Jul 23 15:51:19.853538 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.853499 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:51:19.854024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.853566 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:51:19.854024 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.853663 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:51:19.856327 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.856303 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 15:51:19.856327 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.856311 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf69f\" (UniqueName: \"kubernetes.io/projected/e15e83d8-73c2-47c3-95f3-6763d5f5114f-kube-api-access-mf69f\") pod \"network-check-target-tgdf5\" (UID: \"e15e83d8-73c2-47c3-95f3-6763d5f5114f\") " pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:51:19.856490 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.856303 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 15:51:19.866679 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.866654 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d199f478-aba8-452c-9224-ed736a726a29-metrics-certs\") pod \"network-metrics-daemon-sg68n\" (UID: \"d199f478-aba8-452c-9224-ed736a726a29\") " pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:51:19.866679 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.866674 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b38c39bd-bd8b-40e9-95ad-c1ef17a463a0-original-pull-secret\") pod \"global-pull-secret-syncer-gphkh\" (UID: \"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0\") " pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:51:19.954010 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.953965 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gphkh" Jul 23 15:51:19.963745 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.963719 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xglcr\"" Jul 23 15:51:19.971871 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:19.971844 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:51:20.156623 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:20.156548 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9k2x\"" Jul 23 15:51:20.164768 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:20.164744 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sg68n" Jul 23 15:51:20.235595 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:20.235562 2547 scope.go:117] "RemoveContainer" containerID="1ba55ec84caf06f0881505981729c8e48030f9585c9ca9e040300f14a9675f69" Jul 23 15:51:20.907381 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:20.906295 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tgdf5"] Jul 23 15:51:20.909727 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:20.909672 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15e83d8_73c2_47c3_95f3_6763d5f5114f.slice/crio-bf51343e933a1c4a058b7fe8e3ec75d134e315f3926101415791fb6008acb5e5 WatchSource:0}: Error finding container bf51343e933a1c4a058b7fe8e3ec75d134e315f3926101415791fb6008acb5e5: Status 404 returned error can't find the container with id bf51343e933a1c4a058b7fe8e3ec75d134e315f3926101415791fb6008acb5e5 Jul 23 15:51:20.921886 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:20.921862 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gphkh"] Jul 23 15:51:20.926191 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:20.926154 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb38c39bd_bd8b_40e9_95ad_c1ef17a463a0.slice/crio-a9ff22b8f4ac0863dbe1fcf45e2d1c82ffe2778ded4926b6df2638008e8b5a8e WatchSource:0}: Error finding container a9ff22b8f4ac0863dbe1fcf45e2d1c82ffe2778ded4926b6df2638008e8b5a8e: Status 404 returned error can't find the container with id a9ff22b8f4ac0863dbe1fcf45e2d1c82ffe2778ded4926b6df2638008e8b5a8e Jul 23 15:51:20.939323 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:20.939296 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sg68n"] Jul 23 15:51:20.947824 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:20.947788 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd199f478_aba8_452c_9224_ed736a726a29.slice/crio-59029279ef81b9a56da2185d8d3a2095839bc6557a55aba835be9ccce5f5dd57 WatchSource:0}: Error finding container 59029279ef81b9a56da2185d8d3a2095839bc6557a55aba835be9ccce5f5dd57: Status 404 returned error can't find the container with id 59029279ef81b9a56da2185d8d3a2095839bc6557a55aba835be9ccce5f5dd57 Jul 23 15:51:21.064780 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.064755 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:51:21.064942 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.064789 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:51:21.064942 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.064815 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:51:21.064942 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.064868 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:51:21.064942 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.064897 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:51:21.067328 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.067299 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a3d81a0-7174-498a-be1d-c558568110ee-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ctbdr\" (UID: \"6a3d81a0-7174-498a-be1d-c558568110ee\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:51:21.067421 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.067299 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c85deb5a-7ad1-4904-ab1d-1878dd8e0599-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tsr6z\" (UID: \"c85deb5a-7ad1-4904-ab1d-1878dd8e0599\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:51:21.067569 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.067545 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6-metrics-tls\") pod \"dns-default-ds84x\" (UID: \"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6\") " pod="openshift-dns/dns-default-ds84x" Jul 23 15:51:21.067618 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.067603 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"image-registry-5b8977f69-dwpdp\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:51:21.067661 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.067640 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/561f3cb0-7a0d-4180-b795-504dee568a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4sj5m\" (UID: \"561f3cb0-7a0d-4180-b795-504dee568a8e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:51:21.165519 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.165489 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:21.165645 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.165536 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:21.166099 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.166075 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-service-ca-bundle\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:21.167620 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.167599 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baff5aa4-e1cc-4e0c-baba-6f2711cdaacf-metrics-certs\") pod \"router-default-749854d9c6-jljn4\" (UID: \"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf\") " pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:21.245859 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.245832 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-69gss\"" Jul 23 15:51:21.253749 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.253731 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:51:21.261250 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.261226 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6qz66\"" Jul 23 15:51:21.269355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.269337 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" Jul 23 15:51:21.285416 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.285398 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wtfv7\"" Jul 23 15:51:21.294118 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.294098 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" Jul 23 15:51:21.325546 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.324460 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c4nf5\"" Jul 23 15:51:21.330802 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.330776 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tll2c\"" Jul 23 15:51:21.332865 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.332841 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ds84x" Jul 23 15:51:21.337602 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.337581 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zlj67\"" Jul 23 15:51:21.339473 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.339363 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:21.345305 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.345290 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" Jul 23 15:51:21.424837 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.424758 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8977f69-dwpdp"] Jul 23 15:51:21.447739 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:21.447656 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98002b2a_d733_408d_9dd2_ae98f4273b6d.slice/crio-2078ace2c5922a7ebb90cef3f9aee0104c02383025aa84df821d346ae7e841ff WatchSource:0}: Error finding container 2078ace2c5922a7ebb90cef3f9aee0104c02383025aa84df821d346ae7e841ff: Status 404 returned error can't find the container with id 2078ace2c5922a7ebb90cef3f9aee0104c02383025aa84df821d346ae7e841ff Jul 23 15:51:21.494444 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.494334 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z"] Jul 23 15:51:21.552716 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.552307 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr"] Jul 23 15:51:21.556943 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:21.556812 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3d81a0_7174_498a_be1d_c558568110ee.slice/crio-895c0519e6b13b8aff3811ae2695504a2ed132490fd6f7d5b296c403fc1612c6 WatchSource:0}: Error finding container 895c0519e6b13b8aff3811ae2695504a2ed132490fd6f7d5b296c403fc1612c6: Status 404 returned error can't find the container with id 895c0519e6b13b8aff3811ae2695504a2ed132490fd6f7d5b296c403fc1612c6 Jul 23 15:51:21.590153 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.587018 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" event={"ID":"98002b2a-d733-408d-9dd2-ae98f4273b6d","Type":"ContainerStarted","Data":"aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4"} Jul 23 15:51:21.590153 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.587082 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" event={"ID":"98002b2a-d733-408d-9dd2-ae98f4273b6d","Type":"ContainerStarted","Data":"2078ace2c5922a7ebb90cef3f9aee0104c02383025aa84df821d346ae7e841ff"} Jul 23 15:51:21.597356 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.596814 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tgdf5" event={"ID":"e15e83d8-73c2-47c3-95f3-6763d5f5114f","Type":"ContainerStarted","Data":"defd3c511bb942ffc1d8e2656bd278d274e24522760811ab776236ec4473674e"} Jul 23 15:51:21.597356 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.596881 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tgdf5" event={"ID":"e15e83d8-73c2-47c3-95f3-6763d5f5114f","Type":"ContainerStarted","Data":"bf51343e933a1c4a058b7fe8e3ec75d134e315f3926101415791fb6008acb5e5"} Jul 23 15:51:21.597356 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.596946 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:51:21.600086 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.599324 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ds84x"] Jul 23 15:51:21.602032 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.601618 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 15:51:21.602032 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.601751 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" event={"ID":"d2c43bce-150e-49ca-b1e9-ff80e7521b67","Type":"ContainerStarted","Data":"9dec9ab8b14157a7935478127005e5b55de451f23e2a4e258936da9f342e102e"} Jul 23 15:51:21.604069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.602895 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:51:21.608178 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.607133 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sg68n" event={"ID":"d199f478-aba8-452c-9224-ed736a726a29","Type":"ContainerStarted","Data":"59029279ef81b9a56da2185d8d3a2095839bc6557a55aba835be9ccce5f5dd57"} Jul 23 15:51:21.612923 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.612862 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gphkh" event={"ID":"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0","Type":"ContainerStarted","Data":"a9ff22b8f4ac0863dbe1fcf45e2d1c82ffe2778ded4926b6df2638008e8b5a8e"} Jul 23 15:51:21.613970 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.613926 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" Jul 23 15:51:21.615855 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.615830 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txdfw" event={"ID":"6170b78e-67ba-4e41-9ff7-95b1602719da","Type":"ContainerStarted","Data":"55e32d25435ee4ca4f05b4f96d21bd66049423d041a5381d810ecac68d867588"} Jul 23 15:51:21.617473 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.617452 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" event={"ID":"6a3d81a0-7174-498a-be1d-c558568110ee","Type":"ContainerStarted","Data":"895c0519e6b13b8aff3811ae2695504a2ed132490fd6f7d5b296c403fc1612c6"} Jul 23 15:51:21.624023 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.623979 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tgdf5" podStartSLOduration=67.623963384 podStartE2EDuration="1m7.623963384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:21.612077627 +0000 UTC m=+67.888819030" watchObservedRunningTime="2026-07-23 15:51:21.623963384 +0000 UTC m=+67.900704787" Jul 23 15:51:21.625195 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.625168 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m"] Jul 23 15:51:21.628344 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:21.628315 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561f3cb0_7a0d_4180_b795_504dee568a8e.slice/crio-604f466a38d556f216882b063e1e113aee2c6b2ba90d8ffe8d7eb2b207f21752 WatchSource:0}: Error finding container 604f466a38d556f216882b063e1e113aee2c6b2ba90d8ffe8d7eb2b207f21752: Status 404 returned error can't find the container with id 604f466a38d556f216882b063e1e113aee2c6b2ba90d8ffe8d7eb2b207f21752 Jul 23 15:51:21.630102 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.629865 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-nvzv6" podStartSLOduration=57.553024717 podStartE2EDuration="1m5.629847085s" totalImagesPullingTime="8.076822368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:21.628647132 +0000 UTC m=+67.905388554" watchObservedRunningTime="2026-07-23 15:51:21.629847085 +0000 UTC m=+67.906588465" Jul 23 15:51:21.644774 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:21.644714 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-749854d9c6-jljn4"] Jul 23 15:51:21.648439 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:21.648393 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaff5aa4_e1cc_4e0c_baba_6f2711cdaacf.slice/crio-1f931b26c65a9075df9aec67c97421a2c4d279f4f7e30e0995790de87832d409 WatchSource:0}: Error finding container 1f931b26c65a9075df9aec67c97421a2c4d279f4f7e30e0995790de87832d409: Status 404 returned error can't find the container with id 1f931b26c65a9075df9aec67c97421a2c4d279f4f7e30e0995790de87832d409 Jul 23 15:51:22.625324 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.625180 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-749854d9c6-jljn4" event={"ID":"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf","Type":"ContainerStarted","Data":"5902a01ba7caaa2f4aad32cd44531c9973f170e8bb34699bf4a86714cfb5edf9"} Jul 23 15:51:22.625324 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.625241 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-749854d9c6-jljn4" event={"ID":"baff5aa4-e1cc-4e0c-baba-6f2711cdaacf","Type":"ContainerStarted","Data":"1f931b26c65a9075df9aec67c97421a2c4d279f4f7e30e0995790de87832d409"} Jul 23 15:51:22.627902 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.627874 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" event={"ID":"c85deb5a-7ad1-4904-ab1d-1878dd8e0599","Type":"ContainerStarted","Data":"3511964eaad202bc19314f9d961d5eda0327cdd9073658e655d0d46891aa7de0"} Jul 23 15:51:22.629867 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.629840 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" event={"ID":"561f3cb0-7a0d-4180-b795-504dee568a8e","Type":"ContainerStarted","Data":"604f466a38d556f216882b063e1e113aee2c6b2ba90d8ffe8d7eb2b207f21752"} Jul 23 15:51:22.633415 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.632548 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ds84x" event={"ID":"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6","Type":"ContainerStarted","Data":"4290956e9d41cd80bcb1d93ad34bafd099b3191abf58f4f8ebb14c4c38c63946"} Jul 23 15:51:22.633415 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.633363 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:51:22.646887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.645355 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-749854d9c6-jljn4" podStartSLOduration=66.645337264 podStartE2EDuration="1m6.645337264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:22.644201335 +0000 UTC m=+68.920942737" watchObservedRunningTime="2026-07-23 15:51:22.645337264 +0000 UTC m=+68.922078667" Jul 23 15:51:22.671149 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:22.670788 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" podStartSLOduration=68.670768342 podStartE2EDuration="1m8.670768342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:22.669271078 +0000 UTC m=+68.946012533" watchObservedRunningTime="2026-07-23 15:51:22.670768342 +0000 UTC m=+68.947509745" Jul 23 15:51:23.341369 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:23.340001 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:23.343464 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:23.343249 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:23.639008 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:23.638896 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:23.639998 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:23.639974 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-749854d9c6-jljn4" Jul 23 15:51:30.674396 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.673653 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gphkh" event={"ID":"b38c39bd-bd8b-40e9-95ad-c1ef17a463a0","Type":"ContainerStarted","Data":"d463e06ed7ce5b0978c6eb36c45606b265077af184c769e56ae883b9824e8087"} Jul 23 15:51:30.684005 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.683412 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" event={"ID":"561f3cb0-7a0d-4180-b795-504dee568a8e","Type":"ContainerStarted","Data":"4ed792773a6f369d2bb3f81cd27f40673f010f661b5137123f03ecb893a4ed3c"} Jul 23 15:51:30.688775 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.688749 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ds84x" event={"ID":"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6","Type":"ContainerStarted","Data":"41954fd1d6d3c8265b47d4bfbe3fc958acd7b1a36bf7457c1091b51bc36f13da"} Jul 23 15:51:30.696565 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.696524 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gphkh" podStartSLOduration=67.173871064 podStartE2EDuration="1m16.696511911s" totalImagesPullingTime="9.522640847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:30.696090981 +0000 UTC m=+76.972832384" watchObservedRunningTime="2026-07-23 15:51:30.696511911 +0000 UTC m=+76.973253310" Jul 23 15:51:30.699359 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.699297 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txdfw" event={"ID":"6170b78e-67ba-4e41-9ff7-95b1602719da","Type":"ContainerStarted","Data":"055eacba6d4e9b44e6c588f97df145eeed90763b34233f443b4b0f6c8ef66744"} Jul 23 15:51:30.701806 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.701482 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" event={"ID":"6a3d81a0-7174-498a-be1d-c558568110ee","Type":"ContainerStarted","Data":"013c06b01ea6f29acdde8ab3b71305fa98df83a7799c086565afc1b3c6fdea55"} Jul 23 15:51:30.712725 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.711347 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sg68n" event={"ID":"d199f478-aba8-452c-9224-ed736a726a29","Type":"ContainerStarted","Data":"e202419425a3e3875454476d0229d113dc419f6010fd0631f76dfb89a4c9cade"} Jul 23 15:51:30.720341 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.720104 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" event={"ID":"c85deb5a-7ad1-4904-ab1d-1878dd8e0599","Type":"ContainerStarted","Data":"0052c5a2beaea99f319aa9939a46f9e6c163b260dcd0423cec60d5ad1e0631c9"} Jul 23 15:51:30.740829 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.738757 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ctbdr" podStartSLOduration=65.848403987 podStartE2EDuration="1m14.738736566s" totalImagesPullingTime="8.890332579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:30.737494996 +0000 UTC m=+77.014236397" watchObservedRunningTime="2026-07-23 15:51:30.738736566 +0000 UTC m=+77.015477967" Jul 23 15:51:30.740829 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:30.738931 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4sj5m" podStartSLOduration=51.920826559 podStartE2EDuration="1m0.738921968s" totalImagesPullingTime="8.818095409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:30.71309347 +0000 UTC m=+76.989834883" watchObservedRunningTime="2026-07-23 15:51:30.738921968 +0000 UTC m=+77.015663370" Jul 23 15:51:31.728846 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:31.728805 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sg68n" event={"ID":"d199f478-aba8-452c-9224-ed736a726a29","Type":"ContainerStarted","Data":"aa2bbcb97627a4903289dbe5a3f95e89bff0a56631c270e78974829b2b126a86"} Jul 23 15:51:31.730991 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:31.730955 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" event={"ID":"c85deb5a-7ad1-4904-ab1d-1878dd8e0599","Type":"ContainerStarted","Data":"ebe50228ecccdf3a7e2d46404082585b2a8368ca65d1d914604ce841ce77dd79"} Jul 23 15:51:31.733014 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:31.732984 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ds84x" event={"ID":"2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6","Type":"ContainerStarted","Data":"cfc71833e2463dc35da8dca0ea48ec86771a29038bc542214310961ae6d4f39e"} Jul 23 15:51:31.733199 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:31.733168 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ds84x" Jul 23 15:51:31.744808 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:31.744764 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sg68n" podStartSLOduration=69.497319798 podStartE2EDuration="1m17.744752226s" totalImagesPullingTime="8.247432428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:31.743803504 +0000 UTC m=+78.020544917" watchObservedRunningTime="2026-07-23 15:51:31.744752226 +0000 UTC m=+78.021493626" Jul 23 15:51:31.762682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:31.762640 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tsr6z" podStartSLOduration=66.923137299 podStartE2EDuration="1m15.762623722s" totalImagesPullingTime="8.839486423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:31.760804629 +0000 UTC m=+78.037546055" watchObservedRunningTime="2026-07-23 15:51:31.762623722 +0000 UTC m=+78.039365122" Jul 23 15:51:31.779359 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:31.779320 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ds84x" podStartSLOduration=33.936880322 podStartE2EDuration="42.779307592s" totalImagesPullingTime="8.84242727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:31.778073165 +0000 UTC m=+78.054814566" watchObservedRunningTime="2026-07-23 15:51:31.779307592 +0000 UTC m=+78.056048993" Jul 23 15:51:34.749670 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:34.749636 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-txdfw" event={"ID":"6170b78e-67ba-4e41-9ff7-95b1602719da","Type":"ContainerStarted","Data":"406bf2b285ea34291efc56dcfb67fdbb29ba4e40739caccc65b36f068d7efed4"} Jul 23 15:51:34.768662 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:34.768616 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-txdfw" podStartSLOduration=18.010402672 podStartE2EDuration="35.768601228s" totalImagesPullingTime="17.758198556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 15:50:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:34.767746802 +0000 UTC m=+81.044488203" watchObservedRunningTime="2026-07-23 15:51:34.768601228 +0000 UTC m=+81.045342628" Jul 23 15:51:41.258856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.258744 2547 patch_prober.go:28] interesting pod/image-registry-5b8977f69-dwpdp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 15:51:41.258856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.258807 2547 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" podUID="98002b2a-d733-408d-9dd2-ae98f4273b6d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:51:41.521625 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.521551 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-j5bv2"] Jul 23 15:51:41.524342 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.524322 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.528236 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.528213 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mnr8l\"" Jul 23 15:51:41.528352 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.528257 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 15:51:41.528404 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.528349 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 23 15:51:41.528404 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.528373 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 23 15:51:41.528489 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.528377 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 23 15:51:41.533340 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.533322 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqfwc\" (UniqueName: \"kubernetes.io/projected/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-api-access-bqfwc\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.533435 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.533359 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e79829f6-6fb4-429b-a94f-0c519dae8c76-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.533435 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.533375 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e79829f6-6fb4-429b-a94f-0c519dae8c76-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.533435 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.533406 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.533530 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.533439 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.533530 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.533502 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.536804 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.536785 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-j5bv2"] Jul 23 15:51:41.556485 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.556453 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sxvbj"] Jul 23 15:51:41.563560 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.563539 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.566071 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.566051 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 15:51:41.566173 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.566069 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dbzlp\"" Jul 23 15:51:41.566173 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.566077 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 15:51:41.566343 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.566328 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 15:51:41.634056 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634026 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.634187 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634063 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-wtmp\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634187 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634090 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqfwc\" (UniqueName: \"kubernetes.io/projected/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-api-access-bqfwc\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.634187 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634113 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4nbq\" (UniqueName: \"kubernetes.io/projected/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-kube-api-access-m4nbq\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634187 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634141 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-textfile\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634187 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634162 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-sys\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634204 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-tls\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634240 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-root\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634271 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e79829f6-6fb4-429b-a94f-0c519dae8c76-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.634355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634293 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e79829f6-6fb4-429b-a94f-0c519dae8c76-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.634355 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634315 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.634547 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634418 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-metrics-client-ca\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634547 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634453 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634547 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634520 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.634687 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634555 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-accelerators-collector-config\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.634687 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.634638 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e79829f6-6fb4-429b-a94f-0c519dae8c76-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.635032 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.635009 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e79829f6-6fb4-429b-a94f-0c519dae8c76-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.635291 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.635267 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.636666 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.636643 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.636666 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.636657 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.641422 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.641404 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqfwc\" (UniqueName: \"kubernetes.io/projected/e79829f6-6fb4-429b-a94f-0c519dae8c76-kube-api-access-bqfwc\") pod \"kube-state-metrics-56c5594674-j5bv2\" (UID: \"e79829f6-6fb4-429b-a94f-0c519dae8c76\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.735089 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735059 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-wtmp\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735229 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735095 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4nbq\" (UniqueName: \"kubernetes.io/projected/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-kube-api-access-m4nbq\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735229 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735121 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-textfile\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735229 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735219 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-wtmp\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735394 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735262 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-sys\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735394 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735225 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-sys\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735394 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735295 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-tls\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735394 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735320 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-root\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735394 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:41.735356 2547 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 15:51:41.735590 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735362 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-metrics-client-ca\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735590 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:51:41.735421 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-tls podName:8f42f8fa-883a-46be-ba8b-f5cc32722ac5 nodeName:}" failed. No retries permitted until 2026-07-23 15:51:42.235402999 +0000 UTC m=+88.512144382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-tls") pod "node-exporter-sxvbj" (UID: "8f42f8fa-883a-46be-ba8b-f5cc32722ac5") : secret "node-exporter-tls" not found Jul 23 15:51:41.735590 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735429 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-textfile\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735590 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735431 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-root\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735590 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735462 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735590 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735506 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-accelerators-collector-config\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.735923 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735894 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-metrics-client-ca\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.736002 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.735962 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-accelerators-collector-config\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.737727 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.737710 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.740141 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.740122 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ds84x" Jul 23 15:51:41.746556 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.746537 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4nbq\" (UniqueName: \"kubernetes.io/projected/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-kube-api-access-m4nbq\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:41.833750 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.833719 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" Jul 23 15:51:41.963387 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:41.963361 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-j5bv2"] Jul 23 15:51:41.966024 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:41.965997 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode79829f6_6fb4_429b_a94f_0c519dae8c76.slice/crio-644c1773299a217b8ca20a51e2fffd1f9ec2ef9aee476bcb75215bc5faa6a0df WatchSource:0}: Error finding container 644c1773299a217b8ca20a51e2fffd1f9ec2ef9aee476bcb75215bc5faa6a0df: Status 404 returned error can't find the container with id 644c1773299a217b8ca20a51e2fffd1f9ec2ef9aee476bcb75215bc5faa6a0df Jul 23 15:51:42.241165 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:42.241063 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-tls\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:42.243361 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:42.243337 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8f42f8fa-883a-46be-ba8b-f5cc32722ac5-node-exporter-tls\") pod \"node-exporter-sxvbj\" (UID: \"8f42f8fa-883a-46be-ba8b-f5cc32722ac5\") " pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:42.472126 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:42.472100 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sxvbj" Jul 23 15:51:42.482873 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:42.482842 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f42f8fa_883a_46be_ba8b_f5cc32722ac5.slice/crio-41b5c3c9ef3fa19138d0943f2c062598b7d602cee5bc130a437638f7f1fad5c7 WatchSource:0}: Error finding container 41b5c3c9ef3fa19138d0943f2c062598b7d602cee5bc130a437638f7f1fad5c7: Status 404 returned error can't find the container with id 41b5c3c9ef3fa19138d0943f2c062598b7d602cee5bc130a437638f7f1fad5c7 Jul 23 15:51:42.778372 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:42.778330 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sxvbj" event={"ID":"8f42f8fa-883a-46be-ba8b-f5cc32722ac5","Type":"ContainerStarted","Data":"41b5c3c9ef3fa19138d0943f2c062598b7d602cee5bc130a437638f7f1fad5c7"} Jul 23 15:51:42.779492 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:42.779467 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" event={"ID":"e79829f6-6fb4-429b-a94f-0c519dae8c76","Type":"ContainerStarted","Data":"644c1773299a217b8ca20a51e2fffd1f9ec2ef9aee476bcb75215bc5faa6a0df"} Jul 23 15:51:43.509790 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.509754 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7"] Jul 23 15:51:43.512624 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.512603 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.515453 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.515427 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5ms79l7hsaci1\"" Jul 23 15:51:43.515453 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.515427 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 15:51:43.515647 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.515577 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 15:51:43.515725 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.515651 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 15:51:43.516305 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.516285 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dwbc9\"" Jul 23 15:51:43.516407 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.516338 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 15:51:43.516507 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.516494 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 15:51:43.525213 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.525192 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7"] Jul 23 15:51:43.550591 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550570 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-tls\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.550730 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550606 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfr9k\" (UniqueName: \"kubernetes.io/projected/8acea5af-dc01-432b-8126-070ab1a30b0b-kube-api-access-cfr9k\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.550730 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550655 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.550730 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550677 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.550730 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550724 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.550959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550758 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.550959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550794 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-grpc-tls\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.550959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.550819 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8acea5af-dc01-432b-8126-070ab1a30b0b-metrics-client-ca\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.642111 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.642080 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:51:43.652106 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652075 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-tls\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.652412 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652393 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfr9k\" (UniqueName: \"kubernetes.io/projected/8acea5af-dc01-432b-8126-070ab1a30b0b-kube-api-access-cfr9k\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.652556 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652528 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.652655 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652591 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.652655 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652619 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.652967 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652666 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.652967 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652763 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-grpc-tls\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.652967 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.652793 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8acea5af-dc01-432b-8126-070ab1a30b0b-metrics-client-ca\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.653656 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.653518 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8acea5af-dc01-432b-8126-070ab1a30b0b-metrics-client-ca\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.655371 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.655344 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-tls\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.656217 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.656165 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.656341 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.656314 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.656485 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.656461 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.656855 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.656834 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-grpc-tls\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.656944 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.656913 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8acea5af-dc01-432b-8126-070ab1a30b0b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.660176 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.660152 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfr9k\" (UniqueName: \"kubernetes.io/projected/8acea5af-dc01-432b-8126-070ab1a30b0b-kube-api-access-cfr9k\") pod \"thanos-querier-7bbc7bb9dd-l5rw7\" (UID: \"8acea5af-dc01-432b-8126-070ab1a30b0b\") " pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.823792 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.823764 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:43.969460 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:43.969427 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7"] Jul 23 15:51:44.794970 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:44.794930 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" event={"ID":"e79829f6-6fb4-429b-a94f-0c519dae8c76","Type":"ContainerStarted","Data":"ba7f8b20d07f34b4f16b42db1e9ea1613cb83fc4c9d4642a59682885ae2c0e7b"} Jul 23 15:51:44.795402 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:44.794999 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" event={"ID":"e79829f6-6fb4-429b-a94f-0c519dae8c76","Type":"ContainerStarted","Data":"96d9e9d7cfcea147d1f4a08c24b5dc84a1241fd5e73af853cc4d01bcab72bb35"} Jul 23 15:51:44.795402 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:44.795025 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" event={"ID":"e79829f6-6fb4-429b-a94f-0c519dae8c76","Type":"ContainerStarted","Data":"d0ddd908b269ac0dcafdccaca762d217db406ae76656a0b682a56b25bb2a25ca"} Jul 23 15:51:44.796770 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:44.796439 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" event={"ID":"8acea5af-dc01-432b-8126-070ab1a30b0b","Type":"ContainerStarted","Data":"655f7d4af3cb5c6fa9a0a1c04a06ca7a2ad9dcfed195594d58383edde0f057fb"} Jul 23 15:51:44.798249 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:44.798228 2547 generic.go:356] "Generic (PLEG): container finished" podID="8f42f8fa-883a-46be-ba8b-f5cc32722ac5" containerID="8e4a16812eb588de4048523430629f8d2c2636c9d04e2c47a1395085bbf334ae" exitCode=0 Jul 23 15:51:44.798357 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:44.798277 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sxvbj" event={"ID":"8f42f8fa-883a-46be-ba8b-f5cc32722ac5","Type":"ContainerDied","Data":"8e4a16812eb588de4048523430629f8d2c2636c9d04e2c47a1395085bbf334ae"} Jul 23 15:51:44.836127 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:44.836080 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-j5bv2" podStartSLOduration=2.015212032 podStartE2EDuration="3.836060371s" totalImagesPullingTime="1.820848339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:44.814999663 +0000 UTC m=+91.091741083" watchObservedRunningTime="2026-07-23 15:51:44.836060371 +0000 UTC m=+91.112801775" Jul 23 15:51:45.804506 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:45.804481 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sxvbj" event={"ID":"8f42f8fa-883a-46be-ba8b-f5cc32722ac5","Type":"ContainerStarted","Data":"ed5d0e47a7f365554bbf219727929bfe66eeeb73d2686ecb200edfa25fbd1c15"} Jul 23 15:51:45.804859 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:45.804541 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sxvbj" event={"ID":"8f42f8fa-883a-46be-ba8b-f5cc32722ac5","Type":"ContainerStarted","Data":"fb479897b355c94d260dedef0d75e7ed9afaf905bbf3071e8c4be2c4e5ca8bcb"} Jul 23 15:51:45.825036 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:45.824992 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sxvbj" podStartSLOduration=3.425851836 podStartE2EDuration="4.824977785s" totalImagesPullingTime="1.306109685s" totalInitContainerRuntime="93.016264ms" isStatefulPod=true podCreationTimestamp="2026-07-23 15:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:45.82364432 +0000 UTC m=+92.100385720" watchObservedRunningTime="2026-07-23 15:51:45.824977785 +0000 UTC m=+92.101719186" Jul 23 15:51:46.811216 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:46.811173 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" event={"ID":"8acea5af-dc01-432b-8126-070ab1a30b0b","Type":"ContainerStarted","Data":"7c4e758a57bc2163bc658736031c030b39bcdd3a93f0f3e614c098b67ccbbb74"} Jul 23 15:51:46.811619 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:46.811261 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" event={"ID":"8acea5af-dc01-432b-8126-070ab1a30b0b","Type":"ContainerStarted","Data":"5128df201c8b32f8aac5c26c574a0205106df0924f945a4d30334f76c03072d7"} Jul 23 15:51:46.811619 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:46.811302 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" event={"ID":"8acea5af-dc01-432b-8126-070ab1a30b0b","Type":"ContainerStarted","Data":"1023dd4eb7de1df5fa4c16aa7b68f03f539bf92e65ec8fca342b1384581326cd"} Jul 23 15:51:47.818157 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.818126 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" event={"ID":"8acea5af-dc01-432b-8126-070ab1a30b0b","Type":"ContainerStarted","Data":"27c3432c46a107fb8f9f8378e5b44ad6fe7aa2ab8118cefb4ef51c1c52fd07ee"} Jul 23 15:51:47.818521 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.818190 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" event={"ID":"8acea5af-dc01-432b-8126-070ab1a30b0b","Type":"ContainerStarted","Data":"3f9b7dffcaf8ecc341dbe0a4425d407524a76f9d009eb16358f7b70d413856dc"} Jul 23 15:51:47.818521 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.818221 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" event={"ID":"8acea5af-dc01-432b-8126-070ab1a30b0b","Type":"ContainerStarted","Data":"e3fcf42a82e8f46c6e3d38f547c7f1e6a98daef6b99fecf2a7437e949a7e2db0"} Jul 23 15:51:47.818521 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.818307 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:47.841168 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.841143 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:51:47.843760 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.843746 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.846825 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.846806 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 15:51:47.847202 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847187 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 15:51:47.847286 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847205 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 15:51:47.847480 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847460 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 15:51:47.847549 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847491 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 15:51:47.847549 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847500 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5vlpkvmomq459\"" Jul 23 15:51:47.847654 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847553 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 15:51:47.847654 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847494 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 15:51:47.847654 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.847580 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 15:51:47.848412 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.848396 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b8b2r\"" Jul 23 15:51:47.848412 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.848405 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 15:51:47.849153 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.849136 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 15:51:47.849239 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.849159 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 15:51:47.849991 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.849974 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 15:51:47.850084 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.850053 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 15:51:47.850222 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.850204 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 15:51:47.850671 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.850635 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 15:51:47.859785 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.859746 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" podStartSLOduration=2.159456603 podStartE2EDuration="4.859732133s" totalImagesPullingTime="2.70027553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:51:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:47.858645958 +0000 UTC m=+94.135387377" watchObservedRunningTime="2026-07-23 15:51:47.859732133 +0000 UTC m=+94.136473536" Jul 23 15:51:47.868362 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.868342 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:51:47.885959 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.885939 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886084 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886065 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886151 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886094 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886210 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886154 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886227 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886247 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886269 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886262 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv7kj\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-kube-api-access-jv7kj\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886375 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886311 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-config\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886375 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886354 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886466 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886384 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886466 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886411 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886569 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886520 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886569 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886547 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886674 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886571 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886674 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886604 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886674 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886625 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886674 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886640 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886839 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886741 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886839 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886759 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.886839 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.886780 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.987993 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.987963 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988099 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988004 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988099 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988024 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988203 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988160 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988207 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988243 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988358 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988269 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988358 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988324 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988358 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988350 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988496 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988385 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988496 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988426 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988496 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988483 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988633 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988510 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988633 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988531 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988633 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988538 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988633 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988567 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988633 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988592 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988633 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988617 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv7kj\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-kube-api-access-jv7kj\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988940 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988646 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-config\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988940 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988678 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988940 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988738 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.988940 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.988921 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.991784 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.990010 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.991784 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.990433 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.991784 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.990718 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.991784 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.991361 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.992154 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.992129 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.992804 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.992382 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.992804 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.992420 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.992972 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.992881 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.992972 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.992915 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.992972 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.992932 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.993552 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.993423 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.993676 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.993593 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.993914 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.993895 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.994167 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.994150 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.994247 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.994224 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.994917 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.994895 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-config\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.995473 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.995455 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:47.997623 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:47.997603 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv7kj\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-kube-api-access-jv7kj\") pod \"prometheus-k8s-0\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:48.154146 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:48.154078 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:51:48.297035 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:48.297011 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:51:48.299204 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:51:48.299165 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ada64da_80fb_485d_9307_193cb08e54ed.slice/crio-fd0e770d7acbc7966d1fc4b9b2f0be8c351ff90f9b0523abcf807c72b38fbf15 WatchSource:0}: Error finding container fd0e770d7acbc7966d1fc4b9b2f0be8c351ff90f9b0523abcf807c72b38fbf15: Status 404 returned error can't find the container with id fd0e770d7acbc7966d1fc4b9b2f0be8c351ff90f9b0523abcf807c72b38fbf15 Jul 23 15:51:48.828903 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:48.828864 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerStarted","Data":"fd0e770d7acbc7966d1fc4b9b2f0be8c351ff90f9b0523abcf807c72b38fbf15"} Jul 23 15:51:49.834884 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:49.834849 2547 generic.go:356] "Generic (PLEG): container finished" podID="1ada64da-80fb-485d-9307-193cb08e54ed" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" exitCode=0 Jul 23 15:51:49.835238 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:49.834947 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde"} Jul 23 15:51:49.986472 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:49.986443 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b8977f69-dwpdp"] Jul 23 15:51:53.640861 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:53.640826 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tgdf5" Jul 23 15:51:53.834682 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:53.834652 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7bbc7bb9dd-l5rw7" Jul 23 15:51:54.860994 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:54.860911 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerStarted","Data":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} Jul 23 15:51:54.861440 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:54.860993 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerStarted","Data":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} Jul 23 15:51:54.861440 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:54.861036 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerStarted","Data":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} Jul 23 15:51:54.861440 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:54.861077 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerStarted","Data":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} Jul 23 15:51:54.861440 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:54.861124 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerStarted","Data":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} Jul 23 15:51:54.861440 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:54.861173 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerStarted","Data":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} Jul 23 15:51:54.890513 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:54.890464 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.8692806920000002 podStartE2EDuration="7.890448057s" totalImagesPullingTime="5.002566665s" totalInitContainerRuntime="18.6007ms" isStatefulPod=false podCreationTimestamp="2026-07-23 15:51:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:51:54.888376222 +0000 UTC m=+101.165117624" watchObservedRunningTime="2026-07-23 15:51:54.890448057 +0000 UTC m=+101.167189456" Jul 23 15:51:58.154838 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:51:58.154807 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:52:09.924034 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:09.923984 2547 generic.go:356] "Generic (PLEG): container finished" podID="a8766364-a437-4a90-9058-e1c2549638dc" containerID="7acd7aeb499c704ae4649d6c58274af350a615506fc9a3b3fb30ae3d35b328a5" exitCode=0 Jul 23 15:52:09.924443 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:09.924056 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sztxq" event={"ID":"a8766364-a437-4a90-9058-e1c2549638dc","Type":"ContainerDied","Data":"7acd7aeb499c704ae4649d6c58274af350a615506fc9a3b3fb30ae3d35b328a5"} Jul 23 15:52:09.924443 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:09.924386 2547 scope.go:117] "RemoveContainer" containerID="7acd7aeb499c704ae4649d6c58274af350a615506fc9a3b3fb30ae3d35b328a5" Jul 23 15:52:09.925603 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:09.925578 2547 generic.go:356] "Generic (PLEG): container finished" podID="6902bf59-cc55-492c-9dad-c3eb6c2ffa4c" containerID="f4e45c3c7d7461c26aba76834d9c886f7122c019c7849b9784997fd97ac65fb8" exitCode=0 Jul 23 15:52:09.925715 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:09.925617 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" event={"ID":"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c","Type":"ContainerDied","Data":"f4e45c3c7d7461c26aba76834d9c886f7122c019c7849b9784997fd97ac65fb8"} Jul 23 15:52:09.925869 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:09.925856 2547 scope.go:117] "RemoveContainer" containerID="f4e45c3c7d7461c26aba76834d9c886f7122c019c7849b9784997fd97ac65fb8" Jul 23 15:52:10.808313 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:10.808282 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-749854d9c6-jljn4_baff5aa4-e1cc-4e0c-baba-6f2711cdaacf/router/0.log" Jul 23 15:52:10.820731 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:10.820685 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sbmkn_65e6104e-d595-4c17-9afd-685efbb4deca/serve-healthcheck-canary/0.log" Jul 23 15:52:10.932096 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:10.932057 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-sztxq" event={"ID":"a8766364-a437-4a90-9058-e1c2549638dc","Type":"ContainerStarted","Data":"9c8779b95c2be165af95179c2c2e9ba6d149bf4be8342ab28f3309e8e3835770"} Jul 23 15:52:10.933933 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:10.933908 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-9p8pf" event={"ID":"6902bf59-cc55-492c-9dad-c3eb6c2ffa4c","Type":"ContainerStarted","Data":"076b02b26f692d4fc4d0a84ac8d5fa3f2a8401cb76deb69bcf59a118c3348223"} Jul 23 15:52:14.950645 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:14.950608 2547 generic.go:356] "Generic (PLEG): container finished" podID="9c75e171-26c0-455f-9380-f4a9d348fcda" containerID="5d9e324e4bdd6cedc1e5c3a83d77c67698c750e0e52c60912f1746c4656eba51" exitCode=0 Jul 23 15:52:14.951030 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:14.950683 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" event={"ID":"9c75e171-26c0-455f-9380-f4a9d348fcda","Type":"ContainerDied","Data":"5d9e324e4bdd6cedc1e5c3a83d77c67698c750e0e52c60912f1746c4656eba51"} Jul 23 15:52:14.951030 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:14.951023 2547 scope.go:117] "RemoveContainer" containerID="5d9e324e4bdd6cedc1e5c3a83d77c67698c750e0e52c60912f1746c4656eba51" Jul 23 15:52:15.006915 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.006872 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" podUID="98002b2a-d733-408d-9dd2-ae98f4273b6d" containerName="registry" containerID="cri-o://aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4" gracePeriod=30 Jul 23 15:52:15.258825 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.258803 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:52:15.328709 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328676 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw7dh\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-kube-api-access-pw7dh\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.328871 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328731 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-installation-pull-secrets\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.328871 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328790 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-bound-sa-token\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.328871 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328813 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-image-registry-private-configuration\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.328871 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328835 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-trusted-ca\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.329090 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328881 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.329090 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328911 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98002b2a-d733-408d-9dd2-ae98f4273b6d-ca-trust-extracted\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.329090 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.328965 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-certificates\") pod \"98002b2a-d733-408d-9dd2-ae98f4273b6d\" (UID: \"98002b2a-d733-408d-9dd2-ae98f4273b6d\") " Jul 23 15:52:15.330675 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.330431 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:52:15.330675 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.330639 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:52:15.331676 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.331651 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:52:15.331823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.331765 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:52:15.331823 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.331785 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-kube-api-access-pw7dh" (OuterVolumeSpecName: "kube-api-access-pw7dh") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "kube-api-access-pw7dh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:52:15.331930 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.331855 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:52:15.331930 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.331918 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:52:15.340500 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.340471 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98002b2a-d733-408d-9dd2-ae98f4273b6d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "98002b2a-d733-408d-9dd2-ae98f4273b6d" (UID: "98002b2a-d733-408d-9dd2-ae98f4273b6d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:52:15.429844 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429800 2547 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-image-registry-private-configuration\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.429844 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429840 2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-trusted-ca\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.429844 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429853 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-tls\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.429844 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429861 2547 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98002b2a-d733-408d-9dd2-ae98f4273b6d-ca-trust-extracted\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.430108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429870 2547 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98002b2a-d733-408d-9dd2-ae98f4273b6d-registry-certificates\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.430108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429878 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw7dh\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-kube-api-access-pw7dh\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.430108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429887 2547 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98002b2a-d733-408d-9dd2-ae98f4273b6d-installation-pull-secrets\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.430108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.429897 2547 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98002b2a-d733-408d-9dd2-ae98f4273b6d-bound-sa-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:52:15.956148 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.956119 2547 generic.go:356] "Generic (PLEG): container finished" podID="98002b2a-d733-408d-9dd2-ae98f4273b6d" containerID="aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4" exitCode=0 Jul 23 15:52:15.956631 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.956165 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" Jul 23 15:52:15.956631 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.956215 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" event={"ID":"98002b2a-d733-408d-9dd2-ae98f4273b6d","Type":"ContainerDied","Data":"aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4"} Jul 23 15:52:15.956631 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.956281 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8977f69-dwpdp" event={"ID":"98002b2a-d733-408d-9dd2-ae98f4273b6d","Type":"ContainerDied","Data":"2078ace2c5922a7ebb90cef3f9aee0104c02383025aa84df821d346ae7e841ff"} Jul 23 15:52:15.956631 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.956312 2547 scope.go:117] "RemoveContainer" containerID="aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4" Jul 23 15:52:15.958383 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.958364 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qsbp4" event={"ID":"9c75e171-26c0-455f-9380-f4a9d348fcda","Type":"ContainerStarted","Data":"ef49fde10cabf14c78a7272397135df4ca368478c396f0eabb083b86ce9a5915"} Jul 23 15:52:15.965975 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.965962 2547 scope.go:117] "RemoveContainer" containerID="aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4" Jul 23 15:52:15.966221 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:52:15.966203 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4\": container with ID starting with aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4 not found: ID does not exist" containerID="aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4" Jul 23 15:52:15.966272 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.966227 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4"} err="failed to get container status \"aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4\": rpc error: code = NotFound desc = could not find container \"aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4\": container with ID starting with aeba33dd851a729eecaa5ae11ab3a915b7770232db5bf6654e8a0b6255ab63f4 not found: ID does not exist" Jul 23 15:52:15.991372 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.991341 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b8977f69-dwpdp"] Jul 23 15:52:15.995911 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:15.995889 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b8977f69-dwpdp"] Jul 23 15:52:16.241278 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:16.241194 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98002b2a-d733-408d-9dd2-ae98f4273b6d" path="/var/lib/kubelet/pods/98002b2a-d733-408d-9dd2-ae98f4273b6d/volumes" Jul 23 15:52:48.155118 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:48.155087 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:52:48.170618 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:48.170598 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:52:49.113018 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:52:49.112993 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:06.225591 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.225503 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:53:06.226050 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.225835 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="prometheus" containerID="cri-o://beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" gracePeriod=600 Jul 23 15:53:06.226050 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.225871 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy" containerID="cri-o://743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" gracePeriod=600 Jul 23 15:53:06.226050 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.225888 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="thanos-sidecar" containerID="cri-o://1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" gracePeriod=600 Jul 23 15:53:06.226050 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.225897 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="config-reloader" containerID="cri-o://457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" gracePeriod=600 Jul 23 15:53:06.226050 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.225946 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-thanos" containerID="cri-o://09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" gracePeriod=600 Jul 23 15:53:06.226050 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.225911 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-web" containerID="cri-o://6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" gracePeriod=600 Jul 23 15:53:06.463224 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.463199 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:06.564854 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.564822 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-config-out\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.564994 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.564864 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-1\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.564994 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.564903 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-db\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.564994 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.564923 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-tls-assets\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.564994 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.564954 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.564994 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.564979 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-trusted-ca-bundle\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565014 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-web-config\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565045 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-2\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565068 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-config\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565107 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-kubelet-serving-ca-bundle\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565132 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-thanos-prometheus-http-client-file\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565260 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565171 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-kube-rbac-proxy\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565543 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565348 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:53:06.565543 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565396 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-metrics-client-ca\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565543 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565426 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-0\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565543 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565437 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:53:06.565543 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565461 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-metrics-client-certs\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565543 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565504 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-grpc-tls\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565543 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565508 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:53:06.565930 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565555 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv7kj\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-kube-api-access-jv7kj\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565930 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565578 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-tls\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565930 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565621 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-serving-certs-ca-bundle\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.565930 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565647 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1ada64da-80fb-485d-9307-193cb08e54ed\" (UID: \"1ada64da-80fb-485d-9307-193cb08e54ed\") " Jul 23 15:53:06.566128 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565946 2547 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.566128 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565965 2547 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.566128 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.565980 2547 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.567403 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.566680 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:53:06.567403 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.567127 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:53:06.567403 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.567208 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:53:06.567730 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.567706 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:53:06.568078 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.567801 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-config-out" (OuterVolumeSpecName: "config-out") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 15:53:06.568078 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.567842 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 15:53:06.568561 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.568535 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.568911 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.568888 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.569005 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.568969 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.569224 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.569187 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:53:06.569428 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.569401 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.570163 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.570130 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-config" (OuterVolumeSpecName: "config") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.570249 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.570229 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.570328 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.570309 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-kube-api-access-jv7kj" (OuterVolumeSpecName: "kube-api-access-jv7kj") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "kube-api-access-jv7kj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 15:53:06.570550 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.570532 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.570772 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.570758 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.579477 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.579454 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-web-config" (OuterVolumeSpecName: "web-config") pod "1ada64da-80fb-485d-9307-193cb08e54ed" (UID: "1ada64da-80fb-485d-9307-193cb08e54ed"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 15:53:06.666732 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666682 2547 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666732 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666725 2547 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-web-config\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666732 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666738 2547 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-config\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666750 2547 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666765 2547 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666778 2547 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-kube-rbac-proxy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666789 2547 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-metrics-client-ca\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666800 2547 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666811 2547 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-metrics-client-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666824 2547 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-grpc-tls\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666836 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv7kj\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-kube-api-access-jv7kj\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666847 2547 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666859 2547 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ada64da-80fb-485d-9307-193cb08e54ed-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666871 2547 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ada64da-80fb-485d-9307-193cb08e54ed-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666883 2547 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-config-out\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666895 2547 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ada64da-80fb-485d-9307-193cb08e54ed-prometheus-k8s-db\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:06.666989 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:06.666905 2547 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ada64da-80fb-485d-9307-193cb08e54ed-tls-assets\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 15:53:07.169108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169071 2547 generic.go:356] "Generic (PLEG): container finished" podID="1ada64da-80fb-485d-9307-193cb08e54ed" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" exitCode=0 Jul 23 15:53:07.169108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169092 2547 generic.go:356] "Generic (PLEG): container finished" podID="1ada64da-80fb-485d-9307-193cb08e54ed" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" exitCode=0 Jul 23 15:53:07.169108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169099 2547 generic.go:356] "Generic (PLEG): container finished" podID="1ada64da-80fb-485d-9307-193cb08e54ed" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" exitCode=0 Jul 23 15:53:07.169108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169107 2547 generic.go:356] "Generic (PLEG): container finished" podID="1ada64da-80fb-485d-9307-193cb08e54ed" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" exitCode=0 Jul 23 15:53:07.169108 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169116 2547 generic.go:356] "Generic (PLEG): container finished" podID="1ada64da-80fb-485d-9307-193cb08e54ed" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" exitCode=0 Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169124 2547 generic.go:356] "Generic (PLEG): container finished" podID="1ada64da-80fb-485d-9307-193cb08e54ed" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" exitCode=0 Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169150 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169151 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169233 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169274 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169296 2547 scope.go:117] "RemoveContainer" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169311 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169355 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169395 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} Jul 23 15:53:07.169516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.169436 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ada64da-80fb-485d-9307-193cb08e54ed","Type":"ContainerDied","Data":"fd0e770d7acbc7966d1fc4b9b2f0be8c351ff90f9b0523abcf807c72b38fbf15"} Jul 23 15:53:07.177807 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.177792 2547 scope.go:117] "RemoveContainer" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.184597 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.184581 2547 scope.go:117] "RemoveContainer" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.192138 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.192122 2547 scope.go:117] "RemoveContainer" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.192672 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.192652 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:53:07.195973 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.195954 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:53:07.199160 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.199144 2547 scope.go:117] "RemoveContainer" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.207375 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.207336 2547 scope.go:117] "RemoveContainer" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.214863 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.214846 2547 scope.go:117] "RemoveContainer" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.219027 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219005 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:53:07.219440 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219424 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="thanos-sidecar" Jul 23 15:53:07.219487 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219444 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="thanos-sidecar" Jul 23 15:53:07.219487 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219468 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="config-reloader" Jul 23 15:53:07.219487 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219478 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="config-reloader" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219492 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="init-config-reloader" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219502 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="init-config-reloader" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219512 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-web" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219521 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-web" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219531 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-thanos" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219539 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-thanos" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219555 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="prometheus" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219564 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="prometheus" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219572 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98002b2a-d733-408d-9dd2-ae98f4273b6d" containerName="registry" Jul 23 15:53:07.219574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219577 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="98002b2a-d733-408d-9dd2-ae98f4273b6d" containerName="registry" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219586 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219591 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219645 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-thanos" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219653 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="prometheus" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219660 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219666 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="config-reloader" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219671 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="thanos-sidecar" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219678 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" containerName="kube-rbac-proxy-web" Jul 23 15:53:07.219887 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.219684 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="98002b2a-d733-408d-9dd2-ae98f4273b6d" containerName="registry" Jul 23 15:53:07.222246 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.222232 2547 scope.go:117] "RemoveContainer" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.222471 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:53:07.222453 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": container with ID starting with 09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f not found: ID does not exist" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.222527 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.222479 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} err="failed to get container status \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": rpc error: code = NotFound desc = could not find container \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": container with ID starting with 09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f not found: ID does not exist" Jul 23 15:53:07.222527 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.222499 2547 scope.go:117] "RemoveContainer" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.222746 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:53:07.222729 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": container with ID starting with 743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788 not found: ID does not exist" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.222796 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.222753 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} err="failed to get container status \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": rpc error: code = NotFound desc = could not find container \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": container with ID starting with 743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788 not found: ID does not exist" Jul 23 15:53:07.222796 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.222769 2547 scope.go:117] "RemoveContainer" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.222991 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:53:07.222974 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": container with ID starting with 6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f not found: ID does not exist" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.223034 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.222995 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} err="failed to get container status \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": rpc error: code = NotFound desc = could not find container \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": container with ID starting with 6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f not found: ID does not exist" Jul 23 15:53:07.223034 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.223007 2547 scope.go:117] "RemoveContainer" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.223269 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:53:07.223251 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": container with ID starting with 1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5 not found: ID does not exist" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.223318 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.223273 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} err="failed to get container status \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": rpc error: code = NotFound desc = could not find container \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": container with ID starting with 1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5 not found: ID does not exist" Jul 23 15:53:07.223318 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.223287 2547 scope.go:117] "RemoveContainer" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.223488 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:53:07.223476 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": container with ID starting with 457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19 not found: ID does not exist" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.223525 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.223491 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} err="failed to get container status \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": rpc error: code = NotFound desc = could not find container \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": container with ID starting with 457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19 not found: ID does not exist" Jul 23 15:53:07.223525 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.223504 2547 scope.go:117] "RemoveContainer" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.223725 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:53:07.223711 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": container with ID starting with beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3 not found: ID does not exist" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.223763 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.223730 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} err="failed to get container status \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": rpc error: code = NotFound desc = could not find container \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": container with ID starting with beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3 not found: ID does not exist" Jul 23 15:53:07.223763 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.223741 2547 scope.go:117] "RemoveContainer" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.224150 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:53:07.224131 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": container with ID starting with f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde not found: ID does not exist" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.224235 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224152 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde"} err="failed to get container status \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": rpc error: code = NotFound desc = could not find container \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": container with ID starting with f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde not found: ID does not exist" Jul 23 15:53:07.224235 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224167 2547 scope.go:117] "RemoveContainer" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.224376 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224360 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} err="failed to get container status \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": rpc error: code = NotFound desc = could not find container \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": container with ID starting with 09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f not found: ID does not exist" Jul 23 15:53:07.224417 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224377 2547 scope.go:117] "RemoveContainer" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.224554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224539 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} err="failed to get container status \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": rpc error: code = NotFound desc = could not find container \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": container with ID starting with 743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788 not found: ID does not exist" Jul 23 15:53:07.224554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224554 2547 scope.go:117] "RemoveContainer" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.224794 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224776 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} err="failed to get container status \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": rpc error: code = NotFound desc = could not find container \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": container with ID starting with 6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f not found: ID does not exist" Jul 23 15:53:07.224794 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224790 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.224866 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.224796 2547 scope.go:117] "RemoveContainer" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.225237 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.225212 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} err="failed to get container status \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": rpc error: code = NotFound desc = could not find container \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": container with ID starting with 1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5 not found: ID does not exist" Jul 23 15:53:07.225237 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.225238 2547 scope.go:117] "RemoveContainer" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.225510 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.225488 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} err="failed to get container status \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": rpc error: code = NotFound desc = could not find container \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": container with ID starting with 457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19 not found: ID does not exist" Jul 23 15:53:07.225510 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.225508 2547 scope.go:117] "RemoveContainer" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.225806 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.225787 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} err="failed to get container status \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": rpc error: code = NotFound desc = could not find container \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": container with ID starting with beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3 not found: ID does not exist" Jul 23 15:53:07.226109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.225807 2547 scope.go:117] "RemoveContainer" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.226109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226045 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde"} err="failed to get container status \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": rpc error: code = NotFound desc = could not find container \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": container with ID starting with f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde not found: ID does not exist" Jul 23 15:53:07.226109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226059 2547 scope.go:117] "RemoveContainer" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.226309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226287 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} err="failed to get container status \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": rpc error: code = NotFound desc = could not find container \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": container with ID starting with 09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f not found: ID does not exist" Jul 23 15:53:07.226354 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226311 2547 scope.go:117] "RemoveContainer" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.226535 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226515 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} err="failed to get container status \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": rpc error: code = NotFound desc = could not find container \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": container with ID starting with 743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788 not found: ID does not exist" Jul 23 15:53:07.226535 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226532 2547 scope.go:117] "RemoveContainer" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.226800 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226777 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} err="failed to get container status \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": rpc error: code = NotFound desc = could not find container \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": container with ID starting with 6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f not found: ID does not exist" Jul 23 15:53:07.226956 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.226799 2547 scope.go:117] "RemoveContainer" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.227056 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227033 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} err="failed to get container status \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": rpc error: code = NotFound desc = could not find container \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": container with ID starting with 1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5 not found: ID does not exist" Jul 23 15:53:07.227149 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227064 2547 scope.go:117] "RemoveContainer" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.227149 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227048 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 15:53:07.227245 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227163 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 15:53:07.227285 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227268 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} err="failed to get container status \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": rpc error: code = NotFound desc = could not find container \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": container with ID starting with 457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19 not found: ID does not exist" Jul 23 15:53:07.227336 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227286 2547 scope.go:117] "RemoveContainer" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.227484 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227464 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 15:53:07.227585 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.227514 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 15:53:07.228082 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.228046 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} err="failed to get container status \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": rpc error: code = NotFound desc = could not find container \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": container with ID starting with beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3 not found: ID does not exist" Jul 23 15:53:07.228178 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.228085 2547 scope.go:117] "RemoveContainer" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.228885 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.228858 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde"} err="failed to get container status \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": rpc error: code = NotFound desc = could not find container \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": container with ID starting with f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde not found: ID does not exist" Jul 23 15:53:07.228982 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.228889 2547 scope.go:117] "RemoveContainer" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.229751 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.229097 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 15:53:07.229751 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.229310 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} err="failed to get container status \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": rpc error: code = NotFound desc = could not find container \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": container with ID starting with 09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f not found: ID does not exist" Jul 23 15:53:07.229751 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.229393 2547 scope.go:117] "RemoveContainer" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.230365 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.229951 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} err="failed to get container status \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": rpc error: code = NotFound desc = could not find container \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": container with ID starting with 743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788 not found: ID does not exist" Jul 23 15:53:07.230365 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.229976 2547 scope.go:117] "RemoveContainer" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.230522 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.230399 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 15:53:07.230522 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.230410 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} err="failed to get container status \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": rpc error: code = NotFound desc = could not find container \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": container with ID starting with 6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f not found: ID does not exist" Jul 23 15:53:07.230522 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.230430 2547 scope.go:117] "RemoveContainer" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.230942 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.230877 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} err="failed to get container status \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": rpc error: code = NotFound desc = could not find container \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": container with ID starting with 1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5 not found: ID does not exist" Jul 23 15:53:07.231041 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.230944 2547 scope.go:117] "RemoveContainer" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.231394 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.231321 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 15:53:07.231634 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.231607 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} err="failed to get container status \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": rpc error: code = NotFound desc = could not find container \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": container with ID starting with 457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19 not found: ID does not exist" Jul 23 15:53:07.231741 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.231639 2547 scope.go:117] "RemoveContainer" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.232087 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.232059 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} err="failed to get container status \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": rpc error: code = NotFound desc = could not find container \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": container with ID starting with beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3 not found: ID does not exist" Jul 23 15:53:07.232087 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.232091 2547 scope.go:117] "RemoveContainer" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.232743 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.232918 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.233132 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.233393 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.233647 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5vlpkvmomq459\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.234004 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.234427 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.234486 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b8b2r\"" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.234638 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde"} err="failed to get container status \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": rpc error: code = NotFound desc = could not find container \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": container with ID starting with f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde not found: ID does not exist" Jul 23 15:53:07.235171 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.234722 2547 scope.go:117] "RemoveContainer" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.236594 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.235136 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} err="failed to get container status \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": rpc error: code = NotFound desc = could not find container \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": container with ID starting with 09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f not found: ID does not exist" Jul 23 15:53:07.236594 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.236136 2547 scope.go:117] "RemoveContainer" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.236594 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.236493 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} err="failed to get container status \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": rpc error: code = NotFound desc = could not find container \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": container with ID starting with 743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788 not found: ID does not exist" Jul 23 15:53:07.236594 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.236518 2547 scope.go:117] "RemoveContainer" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.236873 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.236840 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} err="failed to get container status \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": rpc error: code = NotFound desc = could not find container \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": container with ID starting with 6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f not found: ID does not exist" Jul 23 15:53:07.236873 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.236863 2547 scope.go:117] "RemoveContainer" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.237197 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.237165 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} err="failed to get container status \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": rpc error: code = NotFound desc = could not find container \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": container with ID starting with 1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5 not found: ID does not exist" Jul 23 15:53:07.237197 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.237192 2547 scope.go:117] "RemoveContainer" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.237488 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.237466 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} err="failed to get container status \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": rpc error: code = NotFound desc = could not find container \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": container with ID starting with 457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19 not found: ID does not exist" Jul 23 15:53:07.237574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.237489 2547 scope.go:117] "RemoveContainer" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.237799 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.237777 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} err="failed to get container status \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": rpc error: code = NotFound desc = could not find container \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": container with ID starting with beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3 not found: ID does not exist" Jul 23 15:53:07.237799 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.237802 2547 scope.go:117] "RemoveContainer" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.237933 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.237903 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 15:53:07.238113 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.238056 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde"} err="failed to get container status \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": rpc error: code = NotFound desc = could not find container \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": container with ID starting with f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde not found: ID does not exist" Jul 23 15:53:07.238113 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.238080 2547 scope.go:117] "RemoveContainer" containerID="09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f" Jul 23 15:53:07.238457 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.238410 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f"} err="failed to get container status \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": rpc error: code = NotFound desc = could not find container \"09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f\": container with ID starting with 09043062fd7c12e7eb0990a91c7161ddc14a3375c136e100635880e24c61343f not found: ID does not exist" Jul 23 15:53:07.238533 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.238465 2547 scope.go:117] "RemoveContainer" containerID="743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788" Jul 23 15:53:07.239431 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.239251 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788"} err="failed to get container status \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": rpc error: code = NotFound desc = could not find container \"743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788\": container with ID starting with 743aca6ae9a248c6bf11cbae16e1303b3dd784171edac74d8f6e9f6afe143788 not found: ID does not exist" Jul 23 15:53:07.239431 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.239276 2547 scope.go:117] "RemoveContainer" containerID="6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f" Jul 23 15:53:07.239581 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.239539 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f"} err="failed to get container status \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": rpc error: code = NotFound desc = could not find container \"6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f\": container with ID starting with 6b487000d7a001d345737cea525563c9751fc4f3589946c387f02170a99d0b6f not found: ID does not exist" Jul 23 15:53:07.239581 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.239559 2547 scope.go:117] "RemoveContainer" containerID="1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5" Jul 23 15:53:07.239862 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.239840 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5"} err="failed to get container status \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": rpc error: code = NotFound desc = could not find container \"1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5\": container with ID starting with 1177cb463cecd3beb528352065597d59e27e04fc683b64e68dd0bf3bb16c99f5 not found: ID does not exist" Jul 23 15:53:07.239956 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.239873 2547 scope.go:117] "RemoveContainer" containerID="457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19" Jul 23 15:53:07.240193 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.240163 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19"} err="failed to get container status \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": rpc error: code = NotFound desc = could not find container \"457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19\": container with ID starting with 457def36828bbf90a91500416021395c35787ac369151e5355d0ea15a9d3cc19 not found: ID does not exist" Jul 23 15:53:07.240329 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.240198 2547 scope.go:117] "RemoveContainer" containerID="beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3" Jul 23 15:53:07.240498 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.240473 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3"} err="failed to get container status \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": rpc error: code = NotFound desc = could not find container \"beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3\": container with ID starting with beee27440e8d1a1f9ee622855d94cb2927ea722053143d4eec3bdf98e0c851b3 not found: ID does not exist" Jul 23 15:53:07.240498 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.240493 2547 scope.go:117] "RemoveContainer" containerID="f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde" Jul 23 15:53:07.240641 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.240573 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 15:53:07.240814 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.240780 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde"} err="failed to get container status \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": rpc error: code = NotFound desc = could not find container \"f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde\": container with ID starting with f8c0419cbac6cc24d1f0ed9e77e488142ca929270071ba026726be56111b9cde not found: ID does not exist" Jul 23 15:53:07.241673 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.241655 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:53:07.371678 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371648 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.371846 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371715 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.371846 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371742 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.371846 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371763 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.371846 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371778 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.371846 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371822 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372007 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371841 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372007 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371880 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372007 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371909 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372007 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371931 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372007 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.371999 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372152 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372032 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372152 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372052 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-config\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372152 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372071 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-config-out\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372152 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372088 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-web-config\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372152 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372128 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372154 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wc9v\" (UniqueName: \"kubernetes.io/projected/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-kube-api-access-5wc9v\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372172 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372189 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.372296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.372206 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.473615 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473522 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.473615 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473576 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.473856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473686 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.473856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473748 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.473856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473774 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.473856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473803 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.473856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473825 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473859 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473895 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473934 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.473968 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474002 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474032 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-config\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474058 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-config-out\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474094 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-web-config\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474477 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474121 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474477 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474145 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wc9v\" (UniqueName: \"kubernetes.io/projected/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-kube-api-access-5wc9v\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474477 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474170 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474477 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474198 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474477 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474224 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474477 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474258 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474800 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474505 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.474800 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.474605 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.475104 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.475080 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.475180 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.475133 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.475180 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.475166 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.475776 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.475751 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.476817 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.476684 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.476817 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.476779 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.476972 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.476900 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-config-out\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.477749 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.477727 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.477749 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.477739 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-web-config\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.478065 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.478040 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.478473 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.478444 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.479317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.479291 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-config\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.479317 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.479299 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.479639 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.479625 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.479721 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.479687 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.480032 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.480015 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.483506 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.483487 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wc9v\" (UniqueName: \"kubernetes.io/projected/39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff-kube-api-access-5wc9v\") pod \"prometheus-k8s-0\" (UID: \"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.554046 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.554011 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:53:07.694777 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:07.694750 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 15:53:07.696575 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:53:07.696549 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39bd265e_ab48_4e5a_93ef_07f4bf1fe6ff.slice/crio-08b6db6b813fc1e23477e1e4e7c7570943cd6a7de881f6af241db165e52357cc WatchSource:0}: Error finding container 08b6db6b813fc1e23477e1e4e7c7570943cd6a7de881f6af241db165e52357cc: Status 404 returned error can't find the container with id 08b6db6b813fc1e23477e1e4e7c7570943cd6a7de881f6af241db165e52357cc Jul 23 15:53:08.175308 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:08.175279 2547 generic.go:356] "Generic (PLEG): container finished" podID="39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff" containerID="87f238de4ef337ec5e93ab845b203185d5cdf2140d496314e7a206162bff9b9f" exitCode=0 Jul 23 15:53:08.175463 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:08.175351 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerDied","Data":"87f238de4ef337ec5e93ab845b203185d5cdf2140d496314e7a206162bff9b9f"} Jul 23 15:53:08.175463 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:08.175406 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerStarted","Data":"08b6db6b813fc1e23477e1e4e7c7570943cd6a7de881f6af241db165e52357cc"} Jul 23 15:53:08.241027 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:08.241000 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ada64da-80fb-485d-9307-193cb08e54ed" path="/var/lib/kubelet/pods/1ada64da-80fb-485d-9307-193cb08e54ed/volumes" Jul 23 15:53:09.184502 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:09.184470 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerStarted","Data":"1e2a1227dc5c8ef810aae4f988edc94df2da41345c06118e515df61782322acc"} Jul 23 15:53:09.184670 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:09.184571 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerStarted","Data":"5172667b2acd4ddb86f5f55f276b344ecc4d268acb667b4644f649a71f4f479e"} Jul 23 15:53:09.184670 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:09.184602 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerStarted","Data":"e91ecdef136ca931b4aa4c26d4fe28d74c387c875e3e391213f3eff6c33d04c9"} Jul 23 15:53:09.184670 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:09.184628 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerStarted","Data":"6a069dcaceb0c894eca75c79da09b47cf464e33e767da799ac2df4bd8eee0b83"} Jul 23 15:53:09.184670 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:09.184652 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerStarted","Data":"9a84d5bdfa64e4f7b1ca388d36b0c30ffecde95fa4731877288b15430796a941"} Jul 23 15:53:09.184830 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:09.184674 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff","Type":"ContainerStarted","Data":"7e79469272ccc90eeffa5cbaccd54632b81fbd590c656f6241f802f080bfb12e"} Jul 23 15:53:09.214705 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:09.214637 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.193680343 podStartE2EDuration="2.214609889s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.929546ms" isStatefulPod=false podCreationTimestamp="2026-07-23 15:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:53:09.212138256 +0000 UTC m=+175.488879656" watchObservedRunningTime="2026-07-23 15:53:09.214609889 +0000 UTC m=+175.491351292" Jul 23 15:53:12.554175 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:53:12.554136 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:54:07.554510 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:54:07.554479 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:54:07.571522 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:54:07.571492 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:54:08.432680 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:54:08.432654 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 15:55:14.093124 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:55:14.093099 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 15:55:14.095545 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:55:14.095518 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 15:55:14.099569 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:55:14.099546 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 15:55:14.102096 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:55:14.102073 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 15:55:14.103054 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:55:14.103039 2547 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 15:56:32.611418 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.611341 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8"] Jul 23 15:56:32.614781 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.614765 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.629169 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.629133 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-webhook-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.629296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.629190 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f9bbe05-4189-4121-8e51-89605c902143-manifests\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.629296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.629210 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f9bbe05-4189-4121-8e51-89605c902143-tmp\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.629296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.629226 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.629296 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.629242 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4bmh\" (UniqueName: \"kubernetes.io/projected/9f9bbe05-4189-4121-8e51-89605c902143-kube-api-access-b4bmh\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.646599 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:32.646572 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-vh7z7\" is forbidden: User \"system:node:ip-10-0-139-161.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-161.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vh7z7\"" type="*v1.Secret" Jul 23 15:56:32.647656 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:32.647632 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" is forbidden: User \"system:node:ip-10-0-139-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-161.ec2.internal' and this object" podUID="9f9bbe05-4189-4121-8e51-89605c902143" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.648099 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:32.648071 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-139-161.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-161.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" Jul 23 15:56:32.650560 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:32.650541 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-139-161.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-161.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" Jul 23 15:56:32.650902 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:32.650872 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-161.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-161.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 23 15:56:32.650979 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:32.650907 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-161.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-161.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 23 15:56:32.675952 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.675918 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8"] Jul 23 15:56:32.729754 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.729712 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f9bbe05-4189-4121-8e51-89605c902143-manifests\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.729754 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.729755 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f9bbe05-4189-4121-8e51-89605c902143-tmp\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.729988 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.729776 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.729988 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.729794 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bmh\" (UniqueName: \"kubernetes.io/projected/9f9bbe05-4189-4121-8e51-89605c902143-kube-api-access-b4bmh\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.729988 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.729887 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-webhook-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.730154 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.730132 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f9bbe05-4189-4121-8e51-89605c902143-manifests\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:32.731847 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:32.731828 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f9bbe05-4189-4121-8e51-89605c902143-tmp\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:33.500574 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:33.500542 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 15:56:33.602585 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:33.602552 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 15:56:33.730558 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:33.730524 2547 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 23 15:56:33.730947 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:33.730609 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-webhook-cert podName:9f9bbe05-4189-4121-8e51-89605c902143 nodeName:}" failed. No retries permitted until 2026-07-23 15:56:34.230591821 +0000 UTC m=+380.507333203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-webhook-cert") pod "opendatahub-operator-controller-manager-575b84b7c9-wnfd8" (UID: "9f9bbe05-4189-4121-8e51-89605c902143") : failed to sync secret cache: timed out waiting for the condition Jul 23 15:56:33.730947 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:33.730536 2547 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition Jul 23 15:56:33.730947 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:33.730735 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-apiservice-cert podName:9f9bbe05-4189-4121-8e51-89605c902143 nodeName:}" failed. No retries permitted until 2026-07-23 15:56:34.230714683 +0000 UTC m=+380.507456061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-apiservice-cert") pod "opendatahub-operator-controller-manager-575b84b7c9-wnfd8" (UID: "9f9bbe05-4189-4121-8e51-89605c902143") : failed to sync secret cache: timed out waiting for the condition Jul 23 15:56:33.741027 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:33.741013 2547 projected.go:291] Couldn't get configMap opendatahub/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 23 15:56:33.741074 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:33.741042 2547 projected.go:196] Error preparing data for projected volume kube-api-access-b4bmh for pod opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8: failed to sync configmap cache: timed out waiting for the condition Jul 23 15:56:33.741111 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:33.741082 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9bbe05-4189-4121-8e51-89605c902143-kube-api-access-b4bmh podName:9f9bbe05-4189-4121-8e51-89605c902143 nodeName:}" failed. No retries permitted until 2026-07-23 15:56:34.241070238 +0000 UTC m=+380.517811618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b4bmh" (UniqueName: "kubernetes.io/projected/9f9bbe05-4189-4121-8e51-89605c902143-kube-api-access-b4bmh") pod "opendatahub-operator-controller-manager-575b84b7c9-wnfd8" (UID: "9f9bbe05-4189-4121-8e51-89605c902143") : failed to sync configmap cache: timed out waiting for the condition Jul 23 15:56:34.126333 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.126306 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 15:56:34.214985 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.214957 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vh7z7\"" Jul 23 15:56:34.242187 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.240414 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 15:56:34.242995 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.242960 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:34.243135 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.242998 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bmh\" (UniqueName: \"kubernetes.io/projected/9f9bbe05-4189-4121-8e51-89605c902143-kube-api-access-b4bmh\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:34.243135 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.243102 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-webhook-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:34.246069 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.246042 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:34.246189 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.246081 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4bmh\" (UniqueName: \"kubernetes.io/projected/9f9bbe05-4189-4121-8e51-89605c902143-kube-api-access-b4bmh\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:34.246189 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.246106 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f9bbe05-4189-4121-8e51-89605c902143-webhook-cert\") pod \"opendatahub-operator-controller-manager-575b84b7c9-wnfd8\" (UID: \"9f9bbe05-4189-4121-8e51-89605c902143\") " pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:34.424754 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.424667 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:34.589118 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.589095 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8"] Jul 23 15:56:34.592036 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:56:34.592007 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9bbe05_4189_4121_8e51_89605c902143.slice/crio-3e9f7fb3dcd84e9400c5cc39d2e655fef507d304bf294e44d03e0a51f03ce357 WatchSource:0}: Error finding container 3e9f7fb3dcd84e9400c5cc39d2e655fef507d304bf294e44d03e0a51f03ce357: Status 404 returned error can't find the container with id 3e9f7fb3dcd84e9400c5cc39d2e655fef507d304bf294e44d03e0a51f03ce357 Jul 23 15:56:34.594292 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.594273 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 15:56:34.996735 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:34.996676 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" event={"ID":"9f9bbe05-4189-4121-8e51-89605c902143","Type":"ContainerStarted","Data":"3e9f7fb3dcd84e9400c5cc39d2e655fef507d304bf294e44d03e0a51f03ce357"} Jul 23 15:56:39.016195 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:39.016160 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" event={"ID":"9f9bbe05-4189-4121-8e51-89605c902143","Type":"ContainerStarted","Data":"891613d3ceb20a40136fafc7e0a892a84a0e57622560844813c55027e5670140"} Jul 23 15:56:40.022151 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:40.022117 2547 generic.go:356] "Generic (PLEG): container finished" podID="9f9bbe05-4189-4121-8e51-89605c902143" containerID="891613d3ceb20a40136fafc7e0a892a84a0e57622560844813c55027e5670140" exitCode=0 Jul 23 15:56:40.022572 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:40.022207 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" event={"ID":"9f9bbe05-4189-4121-8e51-89605c902143","Type":"ContainerDied","Data":"891613d3ceb20a40136fafc7e0a892a84a0e57622560844813c55027e5670140"} Jul 23 15:56:41.028809 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:41.028772 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" event={"ID":"9f9bbe05-4189-4121-8e51-89605c902143","Type":"ContainerStarted","Data":"d8b5bd3c1cf99608f881a95702ba588325e834adde60b99345ce6ac7ad5dba6a"} Jul 23 15:56:41.029184 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:41.028848 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:41.058366 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:41.058314 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" podStartSLOduration=4.325160583 podStartE2EDuration="9.05829814s" totalImagesPullingTime="3.724225927s" totalInitContainerRuntime="1.00891163s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:56:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:56:41.056977074 +0000 UTC m=+387.333718500" watchObservedRunningTime="2026-07-23 15:56:41.05829814 +0000 UTC m=+387.335039543" Jul 23 15:56:52.035717 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:52.035669 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-575b84b7c9-wnfd8" Jul 23 15:56:58.439019 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.438990 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zjs8r"] Jul 23 15:56:58.447988 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.447967 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:58.455213 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.455195 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 15:56:58.480105 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.480074 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zjs8r"] Jul 23 15:56:58.556443 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.556412 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a08ad834-0797-497a-8945-776dba2807fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjs8r\" (UID: \"a08ad834-0797-497a-8945-776dba2807fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:58.556601 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.556466 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq5bf\" (UniqueName: \"kubernetes.io/projected/a08ad834-0797-497a-8945-776dba2807fc-kube-api-access-dq5bf\") pod \"model-serving-api-54d89bd5fc-zjs8r\" (UID: \"a08ad834-0797-497a-8945-776dba2807fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:58.657860 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.657834 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a08ad834-0797-497a-8945-776dba2807fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjs8r\" (UID: \"a08ad834-0797-497a-8945-776dba2807fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:58.658016 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.657892 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq5bf\" (UniqueName: \"kubernetes.io/projected/a08ad834-0797-497a-8945-776dba2807fc-kube-api-access-dq5bf\") pod \"model-serving-api-54d89bd5fc-zjs8r\" (UID: \"a08ad834-0797-497a-8945-776dba2807fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:58.658016 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:58.657968 2547 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 23 15:56:58.658130 ip-10-0-139-161 kubenswrapper[2547]: E0723 15:56:58.658038 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a08ad834-0797-497a-8945-776dba2807fc-tls-certs podName:a08ad834-0797-497a-8945-776dba2807fc nodeName:}" failed. No retries permitted until 2026-07-23 15:56:59.158023505 +0000 UTC m=+405.434764884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a08ad834-0797-497a-8945-776dba2807fc-tls-certs") pod "model-serving-api-54d89bd5fc-zjs8r" (UID: "a08ad834-0797-497a-8945-776dba2807fc") : secret "model-serving-api-tls" not found Jul 23 15:56:58.685764 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:58.685733 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq5bf\" (UniqueName: \"kubernetes.io/projected/a08ad834-0797-497a-8945-776dba2807fc-kube-api-access-dq5bf\") pod \"model-serving-api-54d89bd5fc-zjs8r\" (UID: \"a08ad834-0797-497a-8945-776dba2807fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:59.161943 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:59.161906 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a08ad834-0797-497a-8945-776dba2807fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjs8r\" (UID: \"a08ad834-0797-497a-8945-776dba2807fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:59.164485 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:59.164453 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a08ad834-0797-497a-8945-776dba2807fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-zjs8r\" (UID: \"a08ad834-0797-497a-8945-776dba2807fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:59.357562 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:59.357530 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:56:59.526250 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:56:59.526216 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zjs8r"] Jul 23 15:56:59.537794 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:56:59.537763 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda08ad834_0797_497a_8945_776dba2807fc.slice/crio-e586baf8d55b69579b07f6aef81263539c910991fd8b9c4d713d6fa4392f72e9 WatchSource:0}: Error finding container e586baf8d55b69579b07f6aef81263539c910991fd8b9c4d713d6fa4392f72e9: Status 404 returned error can't find the container with id e586baf8d55b69579b07f6aef81263539c910991fd8b9c4d713d6fa4392f72e9 Jul 23 15:57:00.109852 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:00.109805 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" event={"ID":"a08ad834-0797-497a-8945-776dba2807fc","Type":"ContainerStarted","Data":"e586baf8d55b69579b07f6aef81263539c910991fd8b9c4d713d6fa4392f72e9"} Jul 23 15:57:02.120730 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.120672 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" event={"ID":"a08ad834-0797-497a-8945-776dba2807fc","Type":"ContainerStarted","Data":"e698e2db8a835df3877bc98ba6f65c9c41094d223639d0e4d04416ef83ee4aca"} Jul 23 15:57:02.121161 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.120769 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:57:02.222462 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.222394 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" podStartSLOduration=2.701206294 podStartE2EDuration="4.222373865s" totalImagesPullingTime="1.521167571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:57:02.22080017 +0000 UTC m=+408.497541572" watchObservedRunningTime="2026-07-23 15:57:02.222373865 +0000 UTC m=+408.499115267" Jul 23 15:57:02.816910 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.816881 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq"] Jul 23 15:57:02.820075 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.820061 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.826251 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.826227 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 15:57:02.826404 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.826250 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 15:57:02.826404 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.826291 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 15:57:02.826404 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.826227 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 15:57:02.826860 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.826509 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n5sjs\"" Jul 23 15:57:02.826860 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.826533 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 15:57:02.848263 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.848239 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq"] Jul 23 15:57:02.897038 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.897011 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hvmp\" (UniqueName: \"kubernetes.io/projected/131efb8c-9532-4b69-92b0-7369885c28a2-kube-api-access-7hvmp\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.897186 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.897063 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/131efb8c-9532-4b69-92b0-7369885c28a2-manager-config\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.897186 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.897083 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/131efb8c-9532-4b69-92b0-7369885c28a2-metrics-cert\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.897186 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.897134 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131efb8c-9532-4b69-92b0-7369885c28a2-cert\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.998405 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.998365 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hvmp\" (UniqueName: \"kubernetes.io/projected/131efb8c-9532-4b69-92b0-7369885c28a2-kube-api-access-7hvmp\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.998405 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.998410 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/131efb8c-9532-4b69-92b0-7369885c28a2-manager-config\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.998654 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.998427 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/131efb8c-9532-4b69-92b0-7369885c28a2-metrics-cert\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.998654 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.998538 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131efb8c-9532-4b69-92b0-7369885c28a2-cert\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:02.999192 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:02.999167 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/131efb8c-9532-4b69-92b0-7369885c28a2-manager-config\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:03.000856 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:03.000827 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/131efb8c-9532-4b69-92b0-7369885c28a2-metrics-cert\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:03.000954 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:03.000903 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131efb8c-9532-4b69-92b0-7369885c28a2-cert\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:03.015415 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:03.015385 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hvmp\" (UniqueName: \"kubernetes.io/projected/131efb8c-9532-4b69-92b0-7369885c28a2-kube-api-access-7hvmp\") pod \"lws-controller-manager-775bb6b8db-dm4hq\" (UID: \"131efb8c-9532-4b69-92b0-7369885c28a2\") " pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:03.129714 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:03.129635 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:03.285447 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:03.285414 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq"] Jul 23 15:57:03.289058 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:57:03.289035 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131efb8c_9532_4b69_92b0_7369885c28a2.slice/crio-bc28b44e318034138e20922640cdd9df79c98808a2245d15df88950d3ba25f57 WatchSource:0}: Error finding container bc28b44e318034138e20922640cdd9df79c98808a2245d15df88950d3ba25f57: Status 404 returned error can't find the container with id bc28b44e318034138e20922640cdd9df79c98808a2245d15df88950d3ba25f57 Jul 23 15:57:04.131905 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:04.131869 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" event={"ID":"131efb8c-9532-4b69-92b0-7369885c28a2","Type":"ContainerStarted","Data":"bc28b44e318034138e20922640cdd9df79c98808a2245d15df88950d3ba25f57"} Jul 23 15:57:06.143465 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:06.143431 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" event={"ID":"131efb8c-9532-4b69-92b0-7369885c28a2","Type":"ContainerStarted","Data":"1b95ebc80c759ab6d92f97a8792db35aeb0b83a9d83df7d7a1ebbe3231871485"} Jul 23 15:57:06.143962 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:06.143529 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:57:06.226982 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:06.226933 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" podStartSLOduration=1.7511914210000001 podStartE2EDuration="4.226917521s" totalImagesPullingTime="2.4757261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:57:06.22579174 +0000 UTC m=+412.502533152" watchObservedRunningTime="2026-07-23 15:57:06.226917521 +0000 UTC m=+412.503658922" Jul 23 15:57:07.399909 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.399878 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g"] Jul 23 15:57:07.403168 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.403153 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.414571 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.414551 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-b7ncg\"" Jul 23 15:57:07.414714 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.414553 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 15:57:07.445293 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.445265 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g"] Jul 23 15:57:07.543032 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.542999 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543209 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543050 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543209 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543141 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543209 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543188 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543315 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543212 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543315 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543233 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2twpt\" (UniqueName: \"kubernetes.io/projected/5caef1be-fb54-4ec5-ac5d-7709b1efd240-kube-api-access-2twpt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543315 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543258 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543315 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543276 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.543436 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.543312 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.644678 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644649 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.644904 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644720 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.644904 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644748 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.644904 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644768 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.644904 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644803 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2twpt\" (UniqueName: \"kubernetes.io/projected/5caef1be-fb54-4ec5-ac5d-7709b1efd240-kube-api-access-2twpt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.644904 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644856 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.644904 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644893 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.645257 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644916 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.645257 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.644953 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.645257 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.645164 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.645398 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.645305 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.645398 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.645365 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.645505 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.645393 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.645606 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.645587 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.647124 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.647106 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.647224 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.647211 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.668403 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.668346 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5caef1be-fb54-4ec5-ac5d-7709b1efd240-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.668554 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.668530 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2twpt\" (UniqueName: \"kubernetes.io/projected/5caef1be-fb54-4ec5-ac5d-7709b1efd240-kube-api-access-2twpt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4v8c8g\" (UID: \"5caef1be-fb54-4ec5-ac5d-7709b1efd240\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:07.712703 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:07.712673 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:08.022309 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:08.022224 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g"] Jul 23 15:57:08.054064 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:57:08.054029 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5caef1be_fb54_4ec5_ac5d_7709b1efd240.slice/crio-c4361cd312dabea73d23b46eca1ecccfc83c31c680ef23d5b97a972912a6a083 WatchSource:0}: Error finding container c4361cd312dabea73d23b46eca1ecccfc83c31c680ef23d5b97a972912a6a083: Status 404 returned error can't find the container with id c4361cd312dabea73d23b46eca1ecccfc83c31c680ef23d5b97a972912a6a083 Jul 23 15:57:08.154106 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:08.154074 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" event={"ID":"5caef1be-fb54-4ec5-ac5d-7709b1efd240","Type":"ContainerStarted","Data":"c4361cd312dabea73d23b46eca1ecccfc83c31c680ef23d5b97a972912a6a083"} Jul 23 15:57:08.998120 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:08.998086 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk"] Jul 23 15:57:09.002031 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.002005 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.013013 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.012984 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 15:57:09.013142 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.013038 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 15:57:09.013142 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.013071 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 15:57:09.013268 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.012999 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wfd9k\"" Jul 23 15:57:09.023611 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.023588 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk"] Jul 23 15:57:09.160001 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.159905 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.160001 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.159965 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2edd8aa4-d17d-4f43-9433-ba0013329544-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.160262 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.160066 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.160262 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.160103 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.160262 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.160129 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.160262 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.160207 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.160262 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.160241 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k66b\" (UniqueName: \"kubernetes.io/projected/2edd8aa4-d17d-4f43-9433-ba0013329544-kube-api-access-2k66b\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.261023 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.260926 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.261023 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.260981 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2edd8aa4-d17d-4f43-9433-ba0013329544-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.261276 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.261051 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.261276 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.261092 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.261276 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.261117 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.261276 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.261173 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.261276 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.261201 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k66b\" (UniqueName: \"kubernetes.io/projected/2edd8aa4-d17d-4f43-9433-ba0013329544-kube-api-access-2k66b\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.262142 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.262087 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.263969 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.263948 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.263969 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.263963 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2edd8aa4-d17d-4f43-9433-ba0013329544-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.264144 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.264006 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.264144 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.264101 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.290752 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.290724 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2edd8aa4-d17d-4f43-9433-ba0013329544-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.293129 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.293094 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k66b\" (UniqueName: \"kubernetes.io/projected/2edd8aa4-d17d-4f43-9433-ba0013329544-kube-api-access-2k66b\") pod \"istiod-openshift-gateway-94bb8fbfd-gzpsk\" (UID: \"2edd8aa4-d17d-4f43-9433-ba0013329544\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.314877 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.314852 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:09.514186 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:09.514107 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk"] Jul 23 15:57:10.464272 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:57:10.464244 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2edd8aa4_d17d_4f43_9433_ba0013329544.slice/crio-401e77b1dbcf9631e653ab6ad6f41d31559f951f90f4252d1aa1c7233dcdb337 WatchSource:0}: Error finding container 401e77b1dbcf9631e653ab6ad6f41d31559f951f90f4252d1aa1c7233dcdb337: Status 404 returned error can't find the container with id 401e77b1dbcf9631e653ab6ad6f41d31559f951f90f4252d1aa1c7233dcdb337 Jul 23 15:57:10.524673 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:10.524640 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:57:10.524756 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:10.524728 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:57:10.524807 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:10.524759 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:57:11.171415 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:11.171376 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" event={"ID":"2edd8aa4-d17d-4f43-9433-ba0013329544","Type":"ContainerStarted","Data":"401e77b1dbcf9631e653ab6ad6f41d31559f951f90f4252d1aa1c7233dcdb337"} Jul 23 15:57:11.173051 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:11.173003 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" event={"ID":"5caef1be-fb54-4ec5-ac5d-7709b1efd240","Type":"ContainerStarted","Data":"4d4206227f9901c8cc76f9410ead4a9f185e55d618adc136a453daa9a5d746c1"} Jul 23 15:57:11.277318 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:11.277271 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" podStartSLOduration=1.8091484530000002 podStartE2EDuration="4.277251512s" totalImagesPullingTime="2.468103059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:57:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:57:11.275512169 +0000 UTC m=+417.552253573" watchObservedRunningTime="2026-07-23 15:57:11.277251512 +0000 UTC m=+417.553992913" Jul 23 15:57:11.712870 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:11.712835 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:11.718812 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:11.718785 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:12.178381 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:12.178346 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:12.179537 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:12.179511 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4v8c8g" Jul 23 15:57:13.010783 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:13.010745 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:57:13.011032 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:13.010818 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:57:13.128591 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:13.128560 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-zjs8r" Jul 23 15:57:13.183920 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:13.183888 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" event={"ID":"2edd8aa4-d17d-4f43-9433-ba0013329544","Type":"ContainerStarted","Data":"a2e2287f74bd319c37f6723e5e3624680e8f123794bf34f8c564c7970ae8b776"} Jul 23 15:57:13.338976 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:13.338917 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" podStartSLOduration=2.7946904249999998 podStartE2EDuration="5.338897528s" totalImagesPullingTime="2.544207103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:57:13.326177358 +0000 UTC m=+419.602918780" watchObservedRunningTime="2026-07-23 15:57:13.338897528 +0000 UTC m=+419.615638928" Jul 23 15:57:14.189324 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:14.189289 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:14.190647 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:14.190619 2547 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-gzpsk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 15:57:14.190805 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:14.190671 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" podUID="2edd8aa4-d17d-4f43-9433-ba0013329544" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 15:57:15.194113 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:15.194082 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gzpsk" Jul 23 15:57:17.150809 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:57:17.150784 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-775bb6b8db-dm4hq" Jul 23 15:59:36.792998 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.792963 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4"] Jul 23 15:59:36.795121 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.795105 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.801870 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.801850 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z9tfg\"" Jul 23 15:59:36.818561 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.818534 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4"] Jul 23 15:59:36.935274 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935243 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935291 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935319 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmtrp\" (UniqueName: \"kubernetes.io/projected/cd032bd0-d807-489d-8008-47c5bac9f4c2-kube-api-access-hmtrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935341 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935366 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd032bd0-d807-489d-8008-47c5bac9f4c2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935438 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935386 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935605 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935436 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935605 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935493 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:36.935605 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:36.935529 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.036763 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.036831 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.036867 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.036926 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.036965 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmtrp\" (UniqueName: \"kubernetes.io/projected/cd032bd0-d807-489d-8008-47c5bac9f4c2-kube-api-access-hmtrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.036993 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.037036 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd032bd0-d807-489d-8008-47c5bac9f4c2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.037075 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.037109 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.037516 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.037269 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.038172 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.037864 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.038172 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.038003 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.038172 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.038086 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.038347 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.038256 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd032bd0-d807-489d-8008-47c5bac9f4c2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.040471 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.040442 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.040979 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.040960 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.055761 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.055731 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd032bd0-d807-489d-8008-47c5bac9f4c2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.057714 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.057680 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmtrp\" (UniqueName: \"kubernetes.io/projected/cd032bd0-d807-489d-8008-47c5bac9f4c2-kube-api-access-hmtrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-tz5d4\" (UID: \"cd032bd0-d807-489d-8008-47c5bac9f4c2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.105162 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.105135 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:37.268211 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.268187 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4"] Jul 23 15:59:37.270326 ip-10-0-139-161 kubenswrapper[2547]: W0723 15:59:37.270300 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd032bd0_d807_489d_8008_47c5bac9f4c2.slice/crio-f8dffaf6a3d0368224ed5eac7aa7b6827ea7bdd10021d06a2b2f2e596be191ea WatchSource:0}: Error finding container f8dffaf6a3d0368224ed5eac7aa7b6827ea7bdd10021d06a2b2f2e596be191ea: Status 404 returned error can't find the container with id f8dffaf6a3d0368224ed5eac7aa7b6827ea7bdd10021d06a2b2f2e596be191ea Jul 23 15:59:37.272322 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.272291 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:59:37.272384 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.272360 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:59:37.272419 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.272387 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 15:59:37.831684 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.831647 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" event={"ID":"cd032bd0-d807-489d-8008-47c5bac9f4c2","Type":"ContainerStarted","Data":"0a0985405214fe7679591bc46021cc68fa173611ab3a326c5ba443974b5bcfdd"} Jul 23 15:59:37.832068 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.831722 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" event={"ID":"cd032bd0-d807-489d-8008-47c5bac9f4c2","Type":"ContainerStarted","Data":"f8dffaf6a3d0368224ed5eac7aa7b6827ea7bdd10021d06a2b2f2e596be191ea"} Jul 23 15:59:37.861724 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:37.861659 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" podStartSLOduration=1.861644542 podStartE2EDuration="1.861644542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 15:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 15:59:37.859586375 +0000 UTC m=+564.136327776" watchObservedRunningTime="2026-07-23 15:59:37.861644542 +0000 UTC m=+564.138385942" Jul 23 15:59:38.105497 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:38.105415 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:38.110318 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:38.110295 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:38.836290 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:38.836261 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 15:59:38.837109 ip-10-0-139-161 kubenswrapper[2547]: I0723 15:59:38.837091 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 16:00:14.124711 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:00:14.124670 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:00:14.126911 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:00:14.126890 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:00:14.127464 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:00:14.127446 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:00:14.129677 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:00:14.129656 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:01:01.286837 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.286797 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hpkln"] Jul 23 16:01:01.290180 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.290161 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.293296 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.293276 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-snhfh\"" Jul 23 16:01:01.293382 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.293276 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 16:01:01.302872 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.302851 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hpkln"] Jul 23 16:01:01.382860 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.382819 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh2wr\" (UniqueName: \"kubernetes.io/projected/36aa9b41-18eb-409c-a638-1e611b0647a3-kube-api-access-mh2wr\") pod \"payload-processing-5bf47cd988-hpkln\" (UID: \"36aa9b41-18eb-409c-a638-1e611b0647a3\") " pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.383171 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.383142 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36aa9b41-18eb-409c-a638-1e611b0647a3-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hpkln\" (UID: \"36aa9b41-18eb-409c-a638-1e611b0647a3\") " pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.484546 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.484509 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh2wr\" (UniqueName: \"kubernetes.io/projected/36aa9b41-18eb-409c-a638-1e611b0647a3-kube-api-access-mh2wr\") pod \"payload-processing-5bf47cd988-hpkln\" (UID: \"36aa9b41-18eb-409c-a638-1e611b0647a3\") " pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.484753 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.484560 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36aa9b41-18eb-409c-a638-1e611b0647a3-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hpkln\" (UID: \"36aa9b41-18eb-409c-a638-1e611b0647a3\") " pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.485328 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.485310 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36aa9b41-18eb-409c-a638-1e611b0647a3-plugins-config-volume\") pod \"payload-processing-5bf47cd988-hpkln\" (UID: \"36aa9b41-18eb-409c-a638-1e611b0647a3\") " pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.499974 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.499942 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh2wr\" (UniqueName: \"kubernetes.io/projected/36aa9b41-18eb-409c-a638-1e611b0647a3-kube-api-access-mh2wr\") pod \"payload-processing-5bf47cd988-hpkln\" (UID: \"36aa9b41-18eb-409c-a638-1e611b0647a3\") " pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.599902 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.599870 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:01.736224 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:01.736200 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-hpkln"] Jul 23 16:01:01.738549 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:01:01.738517 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36aa9b41_18eb_409c_a638_1e611b0647a3.slice/crio-61020eb6dd43a786000c6aa06347bdae833021ed1a45bace5870dba7dcd2efad WatchSource:0}: Error finding container 61020eb6dd43a786000c6aa06347bdae833021ed1a45bace5870dba7dcd2efad: Status 404 returned error can't find the container with id 61020eb6dd43a786000c6aa06347bdae833021ed1a45bace5870dba7dcd2efad Jul 23 16:01:02.220827 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:02.220796 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" event={"ID":"36aa9b41-18eb-409c-a638-1e611b0647a3","Type":"ContainerStarted","Data":"61020eb6dd43a786000c6aa06347bdae833021ed1a45bace5870dba7dcd2efad"} Jul 23 16:01:06.265646 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:06.265610 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" event={"ID":"36aa9b41-18eb-409c-a638-1e611b0647a3","Type":"ContainerStarted","Data":"22674beacbceef89511cb91b0c83e05eca56a1b3c14a09be80a0ee3581beda05"} Jul 23 16:01:06.266121 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:06.265716 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:08.855916 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:08.855582 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz5d4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": read tcp 10.132.0.2:43810->10.132.0.29:15021: read: connection reset by peer" start-of-body= Jul 23 16:01:08.855916 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:08.855641 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" podUID="cd032bd0-d807-489d-8008-47c5bac9f4c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": read tcp 10.132.0.2:43810->10.132.0.29:15021: read: connection reset by peer" Jul 23 16:01:09.280325 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:09.280245 2547 generic.go:356] "Generic (PLEG): container finished" podID="cd032bd0-d807-489d-8008-47c5bac9f4c2" containerID="0a0985405214fe7679591bc46021cc68fa173611ab3a326c5ba443974b5bcfdd" exitCode=0 Jul 23 16:01:09.280325 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:09.280304 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" event={"ID":"cd032bd0-d807-489d-8008-47c5bac9f4c2","Type":"ContainerDied","Data":"0a0985405214fe7679591bc46021cc68fa173611ab3a326c5ba443974b5bcfdd"} Jul 23 16:01:09.280676 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:09.280663 2547 scope.go:117] "RemoveContainer" containerID="0a0985405214fe7679591bc46021cc68fa173611ab3a326c5ba443974b5bcfdd" Jul 23 16:01:09.281751 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:09.281721 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:01:09.281817 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:09.281804 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:01:09.281868 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:09.281853 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:01:09.304737 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:09.304678 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" podStartSLOduration=4.003951186 podStartE2EDuration="8.304662165s" totalImagesPullingTime="4.300710979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:01:06.29832504 +0000 UTC m=+652.575066441" watchObservedRunningTime="2026-07-23 16:01:09.304662165 +0000 UTC m=+655.581403566" Jul 23 16:01:10.286738 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:10.286685 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" event={"ID":"cd032bd0-d807-489d-8008-47c5bac9f4c2","Type":"ContainerStarted","Data":"296f98bd47a7d65d5cf014379287052e84b4dee8bcad70d25a9d77e33fa3a152"} Jul 23 16:01:11.105714 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:11.105658 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 16:01:11.105889 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:11.105849 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz5d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 23 16:01:11.105937 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:11.105892 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" podUID="cd032bd0-d807-489d-8008-47c5bac9f4c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 23 16:01:12.106032 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:12.105995 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz5d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 23 16:01:12.106420 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:12.106051 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" podUID="cd032bd0-d807-489d-8008-47c5bac9f4c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 23 16:01:13.105517 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:13.105480 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tz5d4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body= Jul 23 16:01:13.105729 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:13.105531 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" podUID="cd032bd0-d807-489d-8008-47c5bac9f4c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" Jul 23 16:01:14.109577 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:14.109544 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 16:01:14.110006 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:14.109912 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 16:01:14.110539 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:14.110523 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tz5d4" Jul 23 16:01:17.272542 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:17.272514 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-hpkln" Jul 23 16:01:43.678567 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.678532 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7"] Jul 23 16:01:43.680863 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.680842 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.683636 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.683610 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 16:01:43.684422 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.684399 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m7bzl\"" Jul 23 16:01:43.684542 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.684457 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 16:01:43.684542 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.684459 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 23 16:01:43.695522 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.695498 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7"] Jul 23 16:01:43.773233 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.773181 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7edf82da-be43-4af5-909a-e523d4351ab3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.773405 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.773255 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.773405 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.773272 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvqgh\" (UniqueName: \"kubernetes.io/projected/7edf82da-be43-4af5-909a-e523d4351ab3-kube-api-access-fvqgh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.773405 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.773294 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.773405 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.773363 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.773405 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.773394 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.874745 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.874679 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.874929 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.874756 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.874929 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.874811 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7edf82da-be43-4af5-909a-e523d4351ab3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.874929 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.874883 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.874929 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.874907 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvqgh\" (UniqueName: \"kubernetes.io/projected/7edf82da-be43-4af5-909a-e523d4351ab3-kube-api-access-fvqgh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.875144 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.874948 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.875196 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.875176 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.875268 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.875243 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.875324 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.875243 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.877217 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.877196 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7edf82da-be43-4af5-909a-e523d4351ab3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.877486 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.877467 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7edf82da-be43-4af5-909a-e523d4351ab3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.883416 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.883391 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvqgh\" (UniqueName: \"kubernetes.io/projected/7edf82da-be43-4af5-909a-e523d4351ab3-kube-api-access-fvqgh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7\" (UID: \"7edf82da-be43-4af5-909a-e523d4351ab3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:43.991860 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:43.991777 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:44.123341 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:44.123309 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7"] Jul 23 16:01:44.126913 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:01:44.126889 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7edf82da_be43_4af5_909a_e523d4351ab3.slice/crio-d9bf4403433193efe948fbc195d14f4a7506c794ce7f37d3bb8f5b5126e3f1f3 WatchSource:0}: Error finding container d9bf4403433193efe948fbc195d14f4a7506c794ce7f37d3bb8f5b5126e3f1f3: Status 404 returned error can't find the container with id d9bf4403433193efe948fbc195d14f4a7506c794ce7f37d3bb8f5b5126e3f1f3 Jul 23 16:01:44.128849 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:44.128829 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 16:01:44.440610 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:44.440568 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" event={"ID":"7edf82da-be43-4af5-909a-e523d4351ab3","Type":"ContainerStarted","Data":"d9bf4403433193efe948fbc195d14f4a7506c794ce7f37d3bb8f5b5126e3f1f3"} Jul 23 16:01:52.480766 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:52.480724 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" event={"ID":"7edf82da-be43-4af5-909a-e523d4351ab3","Type":"ContainerStarted","Data":"69fd17f0a3e341099b95ca103879639fd0b307a7e8457c9e316fe769631f7194"} Jul 23 16:01:52.500474 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:52.500419 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" podStartSLOduration=1.788635743 podStartE2EDuration="9.500402696s" totalImagesPullingTime="7.711766953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:01:52.498557419 +0000 UTC m=+698.775298833" watchObservedRunningTime="2026-07-23 16:01:52.500402696 +0000 UTC m=+698.777144096" Jul 23 16:01:53.992912 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:53.992869 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:53.993343 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:53.992944 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:54.005144 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:54.005122 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:01:54.501393 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:01:54.501365 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7" Jul 23 16:02:15.394500 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.394418 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp"] Jul 23 16:02:15.398026 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.398005 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.400491 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.400475 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 23 16:02:15.410843 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.410823 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp"] Jul 23 16:02:15.466130 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.466095 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.466130 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.466135 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.466367 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.466189 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.466367 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.466228 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwwt9\" (UniqueName: \"kubernetes.io/projected/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-kube-api-access-wwwt9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.466367 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.466314 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.466367 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.466336 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567176 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567139 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567176 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567180 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwwt9\" (UniqueName: \"kubernetes.io/projected/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-kube-api-access-wwwt9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567400 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567234 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567400 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567253 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567400 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567311 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567400 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567339 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567626 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567605 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567664 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567642 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.567780 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.567754 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.569539 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.569520 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.569820 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.569800 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.579117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.579096 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwwt9\" (UniqueName: \"kubernetes.io/projected/3010b92a-1c4e-44c8-bab1-7626e2a3e0d1-kube-api-access-wwwt9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp\" (UID: \"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.708367 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.708272 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:15.841247 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:15.841222 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp"] Jul 23 16:02:15.842848 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:02:15.842818 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3010b92a_1c4e_44c8_bab1_7626e2a3e0d1.slice/crio-22757fdc1e543cf563a180b84894097802bd1689aab6d4ae55008694f21d0765 WatchSource:0}: Error finding container 22757fdc1e543cf563a180b84894097802bd1689aab6d4ae55008694f21d0765: Status 404 returned error can't find the container with id 22757fdc1e543cf563a180b84894097802bd1689aab6d4ae55008694f21d0765 Jul 23 16:02:16.588669 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:16.588628 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" event={"ID":"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1","Type":"ContainerStarted","Data":"534d58faa8f8954bbaad977f0d9571dc0a85b010fb22565362164cee33fcf8c7"} Jul 23 16:02:16.589079 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:16.588720 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" event={"ID":"3010b92a-1c4e-44c8-bab1-7626e2a3e0d1","Type":"ContainerStarted","Data":"22757fdc1e543cf563a180b84894097802bd1689aab6d4ae55008694f21d0765"} Jul 23 16:02:16.619523 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:16.619470 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" podStartSLOduration=1.411351777 podStartE2EDuration="1.61945359s" totalImagesPullingTime="208.101813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:02:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:02:16.615228856 +0000 UTC m=+722.891970257" watchObservedRunningTime="2026-07-23 16:02:16.61945359 +0000 UTC m=+722.896194991" Jul 23 16:02:25.708621 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:25.708586 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:25.709142 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:25.708659 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:25.721425 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:25.721395 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:02:26.646889 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:02:26.646860 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp" Jul 23 16:05:14.156144 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:05:14.156061 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:05:14.156653 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:05:14.156208 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:05:14.158465 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:05:14.158442 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:05:14.158605 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:05:14.158586 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:10:14.186732 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:10:14.186629 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:10:14.187854 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:10:14.187829 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:10:14.189016 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:10:14.188993 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:10:14.190038 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:10:14.190022 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:15:14.217266 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:15:14.217240 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:15:14.219325 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:15:14.219307 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:15:14.219509 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:15:14.219484 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:15:14.221601 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:15:14.221586 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:19:08.091741 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.091619 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5"] Jul 23 16:19:08.095201 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.095185 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.098866 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.098845 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f8d3e728-openshift-default-dockercfg-hhskl\"" Jul 23 16:19:08.130329 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.130306 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5"] Jul 23 16:19:08.254812 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.254782 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-socket\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.254812 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.254818 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-istio-token\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.255008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.254837 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ab19545-6693-483c-aef6-965befbd046e-istiod-ca-cert\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.255008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.254929 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ab19545-6693-483c-aef6-965befbd046e-istio-podinfo\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.255008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.254953 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64pqm\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-kube-api-access-64pqm\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.255008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.254989 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-certs\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.255128 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.255049 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-credential-socket\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.255128 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.255064 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-data\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.255128 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.255097 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-envoy\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.355894 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.355802 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-certs\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.355894 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.355846 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-credential-socket\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.355894 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.355862 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-data\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356165 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.355904 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-envoy\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356165 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.355948 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-socket\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356165 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.355979 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-istio-token\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356165 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356010 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ab19545-6693-483c-aef6-965befbd046e-istiod-ca-cert\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356165 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356049 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ab19545-6693-483c-aef6-965befbd046e-istio-podinfo\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356165 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356067 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64pqm\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-kube-api-access-64pqm\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356497 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356285 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-credential-socket\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356497 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356419 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-socket\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356497 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356464 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-data\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356629 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356522 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-certs\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.356877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.356847 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ab19545-6693-483c-aef6-965befbd046e-istiod-ca-cert\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.358322 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.358304 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-envoy\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.358526 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.358509 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ab19545-6693-483c-aef6-965befbd046e-istio-podinfo\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.365294 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.365277 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-istio-token\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.365500 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.365482 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64pqm\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-kube-api-access-64pqm\") pod \"e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.405662 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.405635 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:08.555890 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.555868 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5"] Jul 23 16:19:08.557954 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:19:08.557907 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab19545_6693_483c_aef6_965befbd046e.slice/crio-eee3bbaf056b33bd158acdb9845fdd0858cbd59c95d5136f03cad652b74d7b05 WatchSource:0}: Error finding container eee3bbaf056b33bd158acdb9845fdd0858cbd59c95d5136f03cad652b74d7b05: Status 404 returned error can't find the container with id eee3bbaf056b33bd158acdb9845fdd0858cbd59c95d5136f03cad652b74d7b05 Jul 23 16:19:08.560160 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.560129 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:08.560231 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.560196 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:08.560231 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:08.560225 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:09.400976 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:09.400948 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" event={"ID":"8ab19545-6693-483c-aef6-965befbd046e","Type":"ContainerStarted","Data":"de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a"} Jul 23 16:19:09.401336 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:09.401000 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" event={"ID":"8ab19545-6693-483c-aef6-965befbd046e","Type":"ContainerStarted","Data":"eee3bbaf056b33bd158acdb9845fdd0858cbd59c95d5136f03cad652b74d7b05"} Jul 23 16:19:09.405872 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:09.405852 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:09.410306 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:09.410276 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:09.422660 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:09.422620 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" podStartSLOduration=1.422604919 podStartE2EDuration="1.422604919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:19:09.421208042 +0000 UTC m=+1735.697949476" watchObservedRunningTime="2026-07-23 16:19:09.422604919 +0000 UTC m=+1735.699346320" Jul 23 16:19:10.406824 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:10.406792 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:10.407952 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:10.407928 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:22.477019 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:22.476984 2547 generic.go:356] "Generic (PLEG): container finished" podID="8ab19545-6693-483c-aef6-965befbd046e" containerID="de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a" exitCode=0 Jul 23 16:19:22.477430 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:22.477056 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" event={"ID":"8ab19545-6693-483c-aef6-965befbd046e","Type":"ContainerDied","Data":"de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a"} Jul 23 16:19:22.477471 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:22.477462 2547 scope.go:117] "RemoveContainer" containerID="de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a" Jul 23 16:19:22.478437 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:22.478412 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:22.478488 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:22.478478 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:22.478524 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:22.478509 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:23.483611 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:23.483583 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" event={"ID":"8ab19545-6693-483c-aef6-965befbd046e","Type":"ContainerStarted","Data":"1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9"} Jul 23 16:19:24.405847 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:24.405802 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:24.406875 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:24.406854 2547 patch_prober.go:28] interesting pod/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 16:19:24.406939 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:24.406904 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 16:19:25.406448 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:25.406416 2547 patch_prober.go:28] interesting pod/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 16:19:25.406866 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:25.406465 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 16:19:25.406866 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:25.406734 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:26.406854 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:26.406814 2547 patch_prober.go:28] interesting pod/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 16:19:26.407287 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:26.406885 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 16:19:27.406685 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:27.406644 2547 patch_prober.go:28] interesting pod/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 16:19:27.406901 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:27.406749 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 16:19:28.409863 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:28.409829 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:28.410901 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:28.410879 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:51.685190 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:51.685150 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5"] Jul 23 16:19:51.685749 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:51.685409 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" containerID="cri-o://1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9" gracePeriod=30 Jul 23 16:19:52.808989 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.808957 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff"] Jul 23 16:19:52.812417 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.812401 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.814900 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.814882 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ad12ff20-gw-openshift-default-dockercfg-2skqc\"" Jul 23 16:19:52.824835 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.824812 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff"] Jul 23 16:19:52.996097 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996069 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-socket\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996110 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-certs\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996139 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-data\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996174 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-credential-socket\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996199 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-podinfo\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996234 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-token\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996260 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4296179-8b50-495b-8f3f-dcf1b597297f-istiod-ca-cert\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996458 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996293 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-envoy\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:52.996458 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:52.996348 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6xfx\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-kube-api-access-f6xfx\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.097914 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.097833 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-token\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.097914 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.097873 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4296179-8b50-495b-8f3f-dcf1b597297f-istiod-ca-cert\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.097919 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-envoy\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.097956 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6xfx\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-kube-api-access-f6xfx\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098006 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-socket\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098045 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-certs\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098087 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-data\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098373 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098125 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-credential-socket\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098373 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098161 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-podinfo\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098539 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098510 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-socket\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098562 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-data\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098634 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-certs\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098799 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098729 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-credential-socket\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.098902 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.098879 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4296179-8b50-495b-8f3f-dcf1b597297f-istiod-ca-cert\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.100382 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.100357 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-envoy\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.100676 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.100656 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-podinfo\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.105862 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.105840 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6xfx\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-kube-api-access-f6xfx\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.106291 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.106275 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-token\") pod \"e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.124399 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.124376 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:53.260173 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.260147 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff"] Jul 23 16:19:53.261949 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:19:53.261914 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4296179_8b50_495b_8f3f_dcf1b597297f.slice/crio-bb222f45c04cb718a5b80b6aad41b2a531a3f4dd0f7e6884ac054140b0b0a3af WatchSource:0}: Error finding container bb222f45c04cb718a5b80b6aad41b2a531a3f4dd0f7e6884ac054140b0b0a3af: Status 404 returned error can't find the container with id bb222f45c04cb718a5b80b6aad41b2a531a3f4dd0f7e6884ac054140b0b0a3af Jul 23 16:19:53.264099 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.264064 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:53.264213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.264140 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:53.264213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.264180 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:19:53.631335 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.631293 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" event={"ID":"d4296179-8b50-495b-8f3f-dcf1b597297f","Type":"ContainerStarted","Data":"93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62"} Jul 23 16:19:53.631594 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.631355 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" event={"ID":"d4296179-8b50-495b-8f3f-dcf1b597297f","Type":"ContainerStarted","Data":"bb222f45c04cb718a5b80b6aad41b2a531a3f4dd0f7e6884ac054140b0b0a3af"} Jul 23 16:19:53.650844 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:53.650789 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" podStartSLOduration=1.650769817 podStartE2EDuration="1.650769817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:19:53.650263134 +0000 UTC m=+1779.927004539" watchObservedRunningTime="2026-07-23 16:19:53.650769817 +0000 UTC m=+1779.927511219" Jul 23 16:19:54.124989 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:54.124959 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:54.129674 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:54.129652 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:54.636121 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:54.636089 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:54.636917 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:54.636902 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:19:56.947047 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:56.946983 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:57.038168 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038132 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ab19545-6693-483c-aef6-965befbd046e-istiod-ca-cert\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038377 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038191 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-certs\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038377 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038212 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-istio-token\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038377 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038238 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-socket\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038377 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038267 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64pqm\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-kube-api-access-64pqm\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038377 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038292 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ab19545-6693-483c-aef6-965befbd046e-istio-podinfo\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038377 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038354 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-data\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038686 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038392 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-credential-socket\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038686 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038442 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-envoy\") pod \"8ab19545-6693-483c-aef6-965befbd046e\" (UID: \"8ab19545-6693-483c-aef6-965befbd046e\") " Jul 23 16:19:57.038686 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038537 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:19:57.038686 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038572 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:19:57.038686 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.038572 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab19545-6693-483c-aef6-965befbd046e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:19:57.039057 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.039008 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.039161 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.039061 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.039161 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.039075 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ab19545-6693-483c-aef6-965befbd046e-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.039276 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.039251 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:19:57.039446 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.039419 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:19:57.040837 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.040810 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:19:57.040837 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.040829 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ab19545-6693-483c-aef6-965befbd046e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:19:57.040965 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.040832 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-kube-api-access-64pqm" (OuterVolumeSpecName: "kube-api-access-64pqm") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "kube-api-access-64pqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:19:57.041229 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.041122 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ab19545-6693-483c-aef6-965befbd046e" (UID: "8ab19545-6693-483c-aef6-965befbd046e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:19:57.139877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.139839 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.139877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.139869 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.139877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.139878 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64pqm\" (UniqueName: \"kubernetes.io/projected/8ab19545-6693-483c-aef6-965befbd046e-kube-api-access-64pqm\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.139877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.139888 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ab19545-6693-483c-aef6-965befbd046e-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.140162 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.139896 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.140162 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.139904 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ab19545-6693-483c-aef6-965befbd046e-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:19:57.651766 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.651732 2547 generic.go:356] "Generic (PLEG): container finished" podID="8ab19545-6693-483c-aef6-965befbd046e" containerID="1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9" exitCode=0 Jul 23 16:19:57.651967 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.651789 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" Jul 23 16:19:57.651967 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.651824 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" event={"ID":"8ab19545-6693-483c-aef6-965befbd046e","Type":"ContainerDied","Data":"1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9"} Jul 23 16:19:57.651967 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.651879 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5" event={"ID":"8ab19545-6693-483c-aef6-965befbd046e","Type":"ContainerDied","Data":"eee3bbaf056b33bd158acdb9845fdd0858cbd59c95d5136f03cad652b74d7b05"} Jul 23 16:19:57.651967 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.651902 2547 scope.go:117] "RemoveContainer" containerID="1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9" Jul 23 16:19:57.663103 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.662805 2547 scope.go:117] "RemoveContainer" containerID="de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a" Jul 23 16:19:57.672802 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.672782 2547 scope.go:117] "RemoveContainer" containerID="1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9" Jul 23 16:19:57.673105 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:19:57.673087 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9\": container with ID starting with 1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9 not found: ID does not exist" containerID="1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9" Jul 23 16:19:57.673168 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.673117 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9"} err="failed to get container status \"1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9\": rpc error: code = NotFound desc = could not find container \"1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9\": container with ID starting with 1263ed82f4ee7962cdb7b027c2b1e99ecfe82078578e04a913265b5753bd9ce9 not found: ID does not exist" Jul 23 16:19:57.673168 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.673135 2547 scope.go:117] "RemoveContainer" containerID="de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a" Jul 23 16:19:57.673338 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:19:57.673320 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a\": container with ID starting with de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a not found: ID does not exist" containerID="de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a" Jul 23 16:19:57.673397 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.673347 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a"} err="failed to get container status \"de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a\": rpc error: code = NotFound desc = could not find container \"de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a\": container with ID starting with de93fc28c2cf129344a62ea21df3eadc50c84fc0880d3c457fc27876dd52868a not found: ID does not exist" Jul 23 16:19:57.674061 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.674040 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5"] Jul 23 16:19:57.677072 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:57.677048 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f8d3e728-openshift-default-66c6bcf49c-f2hs5"] Jul 23 16:19:58.240755 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:19:58.240722 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ab19545-6693-483c-aef6-965befbd046e" path="/var/lib/kubelet/pods/8ab19545-6693-483c-aef6-965befbd046e/volumes" Jul 23 16:20:09.951653 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:09.950178 2547 patch_prober.go:28] interesting pod/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": read tcp 10.132.0.2:49526->10.132.0.34:15021: read: connection reset by peer" start-of-body= Jul 23 16:20:09.951653 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:09.950257 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": read tcp 10.132.0.2:49526->10.132.0.34:15021: read: connection reset by peer" Jul 23 16:20:10.716184 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:10.716152 2547 generic.go:356] "Generic (PLEG): container finished" podID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerID="93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62" exitCode=0 Jul 23 16:20:10.716363 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:10.716224 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" event={"ID":"d4296179-8b50-495b-8f3f-dcf1b597297f","Type":"ContainerDied","Data":"93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62"} Jul 23 16:20:10.716634 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:10.716621 2547 scope.go:117] "RemoveContainer" containerID="93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62" Jul 23 16:20:10.717727 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:10.717675 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:10.717832 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:10.717785 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:10.717832 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:10.717817 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:11.723777 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:11.723743 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" event={"ID":"d4296179-8b50-495b-8f3f-dcf1b597297f","Type":"ContainerStarted","Data":"eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea"} Jul 23 16:20:12.125325 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:12.125285 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:20:12.125533 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:12.125499 2547 patch_prober.go:28] interesting pod/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 23 16:20:12.125617 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:12.125568 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 23 16:20:13.124883 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:13.124839 2547 patch_prober.go:28] interesting pod/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 23 16:20:13.125324 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:13.124900 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 23 16:20:14.125079 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:14.125052 2547 patch_prober.go:28] interesting pod/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 23 16:20:14.125447 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:14.125096 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 23 16:20:14.277105 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:14.277080 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:20:14.279364 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:14.279342 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:20:14.279876 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:14.279858 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:20:14.282287 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:14.282268 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:20:15.125556 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:15.125518 2547 patch_prober.go:28] interesting pod/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 23 16:20:15.125966 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:15.125577 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 23 16:20:16.128546 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:16.128514 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:20:16.128995 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:16.128798 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:20:16.129559 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:16.129541 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:20:35.478450 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:35.478409 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff"] Jul 23 16:20:35.478981 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:35.478620 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" containerID="cri-o://eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea" gracePeriod=30 Jul 23 16:20:36.323139 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.323111 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf"] Jul 23 16:20:36.323482 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.323472 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" Jul 23 16:20:36.323527 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.323484 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" Jul 23 16:20:36.323527 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.323499 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" Jul 23 16:20:36.323527 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.323505 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" Jul 23 16:20:36.323624 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.323570 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" Jul 23 16:20:36.323624 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.323578 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab19545-6693-483c-aef6-965befbd046e" containerName="istio-proxy" Jul 23 16:20:36.326816 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.326796 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.329141 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.329121 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-12b5e3d2-openshift-default-dockercfg-dpdvj\"" Jul 23 16:20:36.339465 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.339446 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf"] Jul 23 16:20:36.412859 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.412827 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74775\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-kube-api-access-74775\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413049 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.412877 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-data\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413049 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.412951 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-envoy\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413154 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.413052 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27eb7104-2da4-41eb-9b79-77c878868965-istiod-ca-cert\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413154 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.413099 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-credential-socket\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413154 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.413125 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27eb7104-2da4-41eb-9b79-77c878868965-istio-podinfo\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413313 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.413164 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-socket\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413313 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.413230 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-istio-token\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.413313 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.413287 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-certs\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.514757 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.514725 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-envoy\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.514791 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27eb7104-2da4-41eb-9b79-77c878868965-istiod-ca-cert\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.514824 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-credential-socket\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.514849 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27eb7104-2da4-41eb-9b79-77c878868965-istio-podinfo\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.514889 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-socket\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.514920 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-istio-token\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.515034 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-certs\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.515146 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74775\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-kube-api-access-74775\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515227 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.515186 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-data\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515731 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.515481 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-socket\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515802 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.515768 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-data\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515802 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.515782 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27eb7104-2da4-41eb-9b79-77c878868965-istiod-ca-cert\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.515905 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.515813 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-certs\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.516072 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.516048 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-credential-socket\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.517678 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.517658 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-envoy\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.518102 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.518082 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27eb7104-2da4-41eb-9b79-77c878868965-istio-podinfo\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.522931 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.522909 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-istio-token\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.523108 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.523081 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74775\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-kube-api-access-74775\") pod \"e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.636682 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.636611 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:36.775410 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.775385 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf"] Jul 23 16:20:36.777247 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:20:36.777219 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27eb7104_2da4_41eb_9b79_77c878868965.slice/crio-3255750238f39513c0b32deb27a865d4d1878510d6ea942da915f2907bccbb41 WatchSource:0}: Error finding container 3255750238f39513c0b32deb27a865d4d1878510d6ea942da915f2907bccbb41: Status 404 returned error can't find the container with id 3255750238f39513c0b32deb27a865d4d1878510d6ea942da915f2907bccbb41 Jul 23 16:20:36.779316 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.779285 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:36.779371 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.779356 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:36.779410 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.779385 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:36.849275 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.849242 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" event={"ID":"27eb7104-2da4-41eb-9b79-77c878868965","Type":"ContainerStarted","Data":"33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d"} Jul 23 16:20:36.849388 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.849307 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" event={"ID":"27eb7104-2da4-41eb-9b79-77c878868965","Type":"ContainerStarted","Data":"3255750238f39513c0b32deb27a865d4d1878510d6ea942da915f2907bccbb41"} Jul 23 16:20:36.869893 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:36.869849 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" podStartSLOduration=0.869833189 podStartE2EDuration="869.833189ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:20:36.867855565 +0000 UTC m=+1823.144596990" watchObservedRunningTime="2026-07-23 16:20:36.869833189 +0000 UTC m=+1823.146574590" Jul 23 16:20:37.637521 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:37.637490 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:37.647718 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:37.642650 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:37.854568 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:37.854537 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:37.855304 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:37.855284 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:40.732827 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.732803 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:20:40.755954 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.755837 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-socket\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.755954 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.755941 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-data\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.755983 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-certs\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756010 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-credential-socket\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756050 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-envoy\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756092 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-token\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756141 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4296179-8b50-495b-8f3f-dcf1b597297f-istiod-ca-cert\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756384 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756168 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-podinfo\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756384 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756193 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6xfx\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-kube-api-access-f6xfx\") pod \"d4296179-8b50-495b-8f3f-dcf1b597297f\" (UID: \"d4296179-8b50-495b-8f3f-dcf1b597297f\") " Jul 23 16:20:40.756384 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756194 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:20:40.756719 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756534 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.756719 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756543 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4296179-8b50-495b-8f3f-dcf1b597297f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:20:40.756914 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756771 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:20:40.756914 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.756845 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:20:40.757404 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.757376 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:20:40.758753 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.758719 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:20:40.759208 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.759179 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:20:40.759304 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.759229 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:20:40.759304 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.759265 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-kube-api-access-f6xfx" (OuterVolumeSpecName: "kube-api-access-f6xfx") pod "d4296179-8b50-495b-8f3f-dcf1b597297f" (UID: "d4296179-8b50-495b-8f3f-dcf1b597297f"). InnerVolumeSpecName "kube-api-access-f6xfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:20:40.857758 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857722 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.857758 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857751 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4296179-8b50-495b-8f3f-dcf1b597297f-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.857758 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857760 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.857986 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857770 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6xfx\" (UniqueName: \"kubernetes.io/projected/d4296179-8b50-495b-8f3f-dcf1b597297f-kube-api-access-f6xfx\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.857986 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857781 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.857986 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857790 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.857986 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857798 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.857986 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.857808 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4296179-8b50-495b-8f3f-dcf1b597297f-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:20:40.871164 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.871130 2547 generic.go:356] "Generic (PLEG): container finished" podID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerID="eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea" exitCode=0 Jul 23 16:20:40.871242 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.871185 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" Jul 23 16:20:40.871242 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.871218 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" event={"ID":"d4296179-8b50-495b-8f3f-dcf1b597297f","Type":"ContainerDied","Data":"eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea"} Jul 23 16:20:40.871314 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.871271 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff" event={"ID":"d4296179-8b50-495b-8f3f-dcf1b597297f","Type":"ContainerDied","Data":"bb222f45c04cb718a5b80b6aad41b2a531a3f4dd0f7e6884ac054140b0b0a3af"} Jul 23 16:20:40.871314 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.871297 2547 scope.go:117] "RemoveContainer" containerID="eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea" Jul 23 16:20:40.881271 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.881254 2547 scope.go:117] "RemoveContainer" containerID="93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62" Jul 23 16:20:40.888802 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.888782 2547 scope.go:117] "RemoveContainer" containerID="eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea" Jul 23 16:20:40.889059 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:20:40.889040 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea\": container with ID starting with eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea not found: ID does not exist" containerID="eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea" Jul 23 16:20:40.889128 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.889069 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea"} err="failed to get container status \"eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea\": rpc error: code = NotFound desc = could not find container \"eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea\": container with ID starting with eca9b4db694eecbf7fb9a18b449171bc575f0f22b75c6432aee94737e032b7ea not found: ID does not exist" Jul 23 16:20:40.889128 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.889087 2547 scope.go:117] "RemoveContainer" containerID="93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62" Jul 23 16:20:40.889334 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:20:40.889318 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62\": container with ID starting with 93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62 not found: ID does not exist" containerID="93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62" Jul 23 16:20:40.889400 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.889339 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62"} err="failed to get container status \"93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62\": rpc error: code = NotFound desc = could not find container \"93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62\": container with ID starting with 93e1ed54bae29961528bf87c47789b4a5b3b59163916565d16871d868d8bbc62 not found: ID does not exist" Jul 23 16:20:40.892980 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.892956 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff"] Jul 23 16:20:40.899389 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:40.899370 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ad12ff20-gw-openshift-default-65ffdcb5f9-2jgff"] Jul 23 16:20:42.241247 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:42.241207 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" path="/var/lib/kubelet/pods/d4296179-8b50-495b-8f3f-dcf1b597297f/volumes" Jul 23 16:20:53.453826 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.453192 2547 patch_prober.go:28] interesting pod/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": read tcp 10.132.0.2:51162->10.132.0.35:15021: read: connection reset by peer" start-of-body= Jul 23 16:20:53.453826 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.453304 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": read tcp 10.132.0.2:51162->10.132.0.35:15021: read: connection reset by peer" Jul 23 16:20:53.937208 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.937176 2547 generic.go:356] "Generic (PLEG): container finished" podID="27eb7104-2da4-41eb-9b79-77c878868965" containerID="33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d" exitCode=0 Jul 23 16:20:53.937404 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.937232 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" event={"ID":"27eb7104-2da4-41eb-9b79-77c878868965","Type":"ContainerDied","Data":"33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d"} Jul 23 16:20:53.937647 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.937630 2547 scope.go:117] "RemoveContainer" containerID="33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d" Jul 23 16:20:53.938657 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.938625 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:53.938733 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.938713 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:53.938781 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:53.938747 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:20:54.948819 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:54.948776 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" event={"ID":"27eb7104-2da4-41eb-9b79-77c878868965","Type":"ContainerStarted","Data":"968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a"} Jul 23 16:20:55.637860 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:55.637829 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:55.638224 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:55.638143 2547 patch_prober.go:28] interesting pod/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 16:20:55.638406 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:55.638380 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 16:20:56.637140 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:56.637099 2547 patch_prober.go:28] interesting pod/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 16:20:56.637770 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:56.637736 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 16:20:57.637834 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:57.637800 2547 patch_prober.go:28] interesting pod/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 16:20:57.638424 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:57.638391 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 16:20:58.638105 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:58.638061 2547 patch_prober.go:28] interesting pod/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 16:20:58.638582 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:58.638141 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 16:20:59.640728 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:59.640673 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:59.641127 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:59.641046 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:20:59.641712 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:20:59.641674 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:21:20.657150 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:20.657115 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf"] Jul 23 16:21:20.657646 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:20.657374 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" containerID="cri-o://968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a" gracePeriod=30 Jul 23 16:21:21.456183 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.456144 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv"] Jul 23 16:21:21.456595 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.456576 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" Jul 23 16:21:21.456595 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.456595 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" Jul 23 16:21:21.456774 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.456607 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" Jul 23 16:21:21.456774 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.456613 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" Jul 23 16:21:21.456774 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.456705 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" Jul 23 16:21:21.456774 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.456716 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4296179-8b50-495b-8f3f-dcf1b597297f" containerName="istio-proxy" Jul 23 16:21:21.459888 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.459868 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.462643 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.462621 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5bb1d507-openshift-default-dockercfg-2fvfv\"" Jul 23 16:21:21.473203 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.473180 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv"] Jul 23 16:21:21.549775 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.549742 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-data\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.549941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.549804 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sssgc\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-kube-api-access-sssgc\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.549941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.549832 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istiod-ca-cert\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.549941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.549865 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-podinfo\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.550079 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.549949 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-certs\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.550079 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.549994 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-socket\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.550079 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.550021 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-credential-socket\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.550079 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.550060 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-token\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.550213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.550099 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-envoy\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.650671 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.650635 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-certs\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.650871 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.650714 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-socket\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.650871 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.650756 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-credential-socket\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.650871 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.650810 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-token\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651075 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651053 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-certs\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651152 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651051 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-envoy\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651152 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651136 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-data\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651247 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651084 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-socket\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651247 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651169 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-credential-socket\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651247 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651194 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sssgc\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-kube-api-access-sssgc\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651247 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651224 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istiod-ca-cert\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651451 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651266 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-podinfo\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.651451 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.651403 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-data\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.652060 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.652023 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istiod-ca-cert\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.653330 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.653305 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-envoy\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.654327 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.654300 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-podinfo\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.659122 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.659101 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-token\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.659632 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.659614 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sssgc\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-kube-api-access-sssgc\") pod \"e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.771940 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.771857 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:21.903063 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.903035 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv"] Jul 23 16:21:21.904676 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:21:21.904651 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28fd700_ddde_4f95_a78c_9f2086e0f9cc.slice/crio-217e00713d0b317f790448ceeaab8db341dea831b263fc42076b32a6780d6196 WatchSource:0}: Error finding container 217e00713d0b317f790448ceeaab8db341dea831b263fc42076b32a6780d6196: Status 404 returned error can't find the container with id 217e00713d0b317f790448ceeaab8db341dea831b263fc42076b32a6780d6196 Jul 23 16:21:21.906952 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.906919 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:21:21.907058 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.906982 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:21:21.907058 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:21.907023 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:21:22.080935 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:22.080900 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" event={"ID":"e28fd700-ddde-4f95-a78c-9f2086e0f9cc","Type":"ContainerStarted","Data":"6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec"} Jul 23 16:21:22.081107 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:22.080961 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" event={"ID":"e28fd700-ddde-4f95-a78c-9f2086e0f9cc","Type":"ContainerStarted","Data":"217e00713d0b317f790448ceeaab8db341dea831b263fc42076b32a6780d6196"} Jul 23 16:21:22.102793 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:22.102739 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" podStartSLOduration=1.102721261 podStartE2EDuration="1.102721261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:21:22.099021333 +0000 UTC m=+1868.375762725" watchObservedRunningTime="2026-07-23 16:21:22.102721261 +0000 UTC m=+1868.379462664" Jul 23 16:21:22.772233 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:22.772195 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:22.777255 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:22.777232 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:23.086379 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:23.086345 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:23.087336 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:23.087318 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:25.908785 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.908749 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:21:25.992113 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992039 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74775\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-kube-api-access-74775\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992113 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992109 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-envoy\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992350 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992143 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-certs\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992350 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992185 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-socket\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992350 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992216 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27eb7104-2da4-41eb-9b79-77c878868965-istiod-ca-cert\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992350 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992244 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-data\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992350 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992298 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-istio-token\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992350 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992334 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-credential-socket\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992679 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992366 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27eb7104-2da4-41eb-9b79-77c878868965-istio-podinfo\") pod \"27eb7104-2da4-41eb-9b79-77c878868965\" (UID: \"27eb7104-2da4-41eb-9b79-77c878868965\") " Jul 23 16:21:25.992679 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992482 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:21:25.992679 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992501 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:21:25.992679 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992581 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27eb7104-2da4-41eb-9b79-77c878868965-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:21:25.992972 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992774 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:21:25.992972 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992834 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:25.992972 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992857 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:25.992972 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.992872 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27eb7104-2da4-41eb-9b79-77c878868965-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:25.993537 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.993513 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-data" (OuterVolumeSpecName: "istio-data") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:21:25.994354 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.994325 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-kube-api-access-74775" (OuterVolumeSpecName: "kube-api-access-74775") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "kube-api-access-74775". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:21:25.994452 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.994429 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:21:25.994578 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.994559 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27eb7104-2da4-41eb-9b79-77c878868965-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:21:25.994616 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:25.994560 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-istio-token" (OuterVolumeSpecName: "istio-token") pod "27eb7104-2da4-41eb-9b79-77c878868965" (UID: "27eb7104-2da4-41eb-9b79-77c878868965"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:21:26.093276 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.093252 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:26.093276 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.093273 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:26.093407 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.093285 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:26.093407 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.093298 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27eb7104-2da4-41eb-9b79-77c878868965-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:26.093407 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.093310 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74775\" (UniqueName: \"kubernetes.io/projected/27eb7104-2da4-41eb-9b79-77c878868965-kube-api-access-74775\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:26.093407 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.093322 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27eb7104-2da4-41eb-9b79-77c878868965-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:21:26.102757 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.102732 2547 generic.go:356] "Generic (PLEG): container finished" podID="27eb7104-2da4-41eb-9b79-77c878868965" containerID="968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a" exitCode=0 Jul 23 16:21:26.102851 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.102766 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" event={"ID":"27eb7104-2da4-41eb-9b79-77c878868965","Type":"ContainerDied","Data":"968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a"} Jul 23 16:21:26.102851 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.102778 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" Jul 23 16:21:26.102851 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.102822 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf" event={"ID":"27eb7104-2da4-41eb-9b79-77c878868965","Type":"ContainerDied","Data":"3255750238f39513c0b32deb27a865d4d1878510d6ea942da915f2907bccbb41"} Jul 23 16:21:26.103005 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.102853 2547 scope.go:117] "RemoveContainer" containerID="968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a" Jul 23 16:21:26.111131 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.111116 2547 scope.go:117] "RemoveContainer" containerID="33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d" Jul 23 16:21:26.118414 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.118399 2547 scope.go:117] "RemoveContainer" containerID="968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a" Jul 23 16:21:26.118654 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:21:26.118636 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a\": container with ID starting with 968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a not found: ID does not exist" containerID="968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a" Jul 23 16:21:26.118727 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.118661 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a"} err="failed to get container status \"968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a\": rpc error: code = NotFound desc = could not find container \"968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a\": container with ID starting with 968d36e4b9c5e64b9db910ab08c732b3f69f7e8d18d793934bea47dc5e3b8b3a not found: ID does not exist" Jul 23 16:21:26.118727 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.118680 2547 scope.go:117] "RemoveContainer" containerID="33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d" Jul 23 16:21:26.118918 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:21:26.118904 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d\": container with ID starting with 33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d not found: ID does not exist" containerID="33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d" Jul 23 16:21:26.118957 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.118924 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d"} err="failed to get container status \"33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d\": rpc error: code = NotFound desc = could not find container \"33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d\": container with ID starting with 33b590b56b8572334a7211b96216aae6cf1d50e37d59e127a43c58afd3353c8d not found: ID does not exist" Jul 23 16:21:26.123582 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.123561 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf"] Jul 23 16:21:26.126672 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.126655 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-12b5e3d2-openshift-default-76f87df8f5-67cvf"] Jul 23 16:21:26.240257 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:26.240231 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27eb7104-2da4-41eb-9b79-77c878868965" path="/var/lib/kubelet/pods/27eb7104-2da4-41eb-9b79-77c878868965/volumes" Jul 23 16:21:38.088038 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.088002 2547 patch_prober.go:28] interesting pod/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 16:21:38.088446 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.088063 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 16:21:38.168225 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.168193 2547 generic.go:356] "Generic (PLEG): container finished" podID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerID="6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec" exitCode=0 Jul 23 16:21:38.168407 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.168271 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" event={"ID":"e28fd700-ddde-4f95-a78c-9f2086e0f9cc","Type":"ContainerDied","Data":"6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec"} Jul 23 16:21:38.168680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.168662 2547 scope.go:117] "RemoveContainer" containerID="6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec" Jul 23 16:21:38.169656 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.169626 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:21:38.169760 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.169706 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:21:38.169760 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:38.169753 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:21:39.175295 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:39.175263 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" event={"ID":"e28fd700-ddde-4f95-a78c-9f2086e0f9cc","Type":"ContainerStarted","Data":"b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62"} Jul 23 16:21:39.772180 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:39.772142 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:39.772424 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:39.772400 2547 patch_prober.go:28] interesting pod/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 16:21:39.772481 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:39.772461 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 16:21:40.773189 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:40.773139 2547 patch_prober.go:28] interesting pod/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 16:21:40.773594 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:40.773221 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 16:21:41.772486 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:41.772446 2547 patch_prober.go:28] interesting pod/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 16:21:41.772665 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:41.772509 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 16:21:42.772962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:42.772928 2547 patch_prober.go:28] interesting pod/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 16:21:42.773359 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:42.772993 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 16:21:43.776372 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:43.776341 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:43.776771 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:43.776666 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:21:43.777471 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:21:43.777454 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:22:03.671110 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:03.671082 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv"] Jul 23 16:22:03.671637 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:03.671278 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" containerID="cri-o://b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62" gracePeriod=30 Jul 23 16:22:08.932178 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:08.932116 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:22:09.001476 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001448 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-socket\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001636 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001484 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-podinfo\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001636 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001501 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-envoy\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001636 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001549 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-credential-socket\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001636 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001594 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istiod-ca-cert\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001636 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001623 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-certs\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001647 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-token\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001727 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sssgc\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-kube-api-access-sssgc\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001764 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-data\") pod \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\" (UID: \"e28fd700-ddde-4f95-a78c-9f2086e0f9cc\") " Jul 23 16:22:09.001941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001770 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:22:09.001941 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001911 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:22:09.002531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.001947 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:22:09.002531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.002345 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:22:09.002531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.002494 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.002531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.002515 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.002531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.002529 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.002831 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.002542 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.002831 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.002809 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:22:09.003953 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.003915 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:22:09.004054 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.003977 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-kube-api-access-sssgc" (OuterVolumeSpecName: "kube-api-access-sssgc") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "kube-api-access-sssgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:22:09.004295 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.004273 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:22:09.004354 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.004283 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "e28fd700-ddde-4f95-a78c-9f2086e0f9cc" (UID: "e28fd700-ddde-4f95-a78c-9f2086e0f9cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:22:09.103883 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.103853 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.103883 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.103878 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.103883 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.103887 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.104095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.103896 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sssgc\" (UniqueName: \"kubernetes.io/projected/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-kube-api-access-sssgc\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.104095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.103904 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28fd700-ddde-4f95-a78c-9f2086e0f9cc-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:22:09.319895 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.319859 2547 generic.go:356] "Generic (PLEG): container finished" podID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerID="b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62" exitCode=0 Jul 23 16:22:09.320056 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.319910 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" Jul 23 16:22:09.320056 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.319943 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" event={"ID":"e28fd700-ddde-4f95-a78c-9f2086e0f9cc","Type":"ContainerDied","Data":"b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62"} Jul 23 16:22:09.320056 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.319998 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv" event={"ID":"e28fd700-ddde-4f95-a78c-9f2086e0f9cc","Type":"ContainerDied","Data":"217e00713d0b317f790448ceeaab8db341dea831b263fc42076b32a6780d6196"} Jul 23 16:22:09.320056 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.320021 2547 scope.go:117] "RemoveContainer" containerID="b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62" Jul 23 16:22:09.329573 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.329554 2547 scope.go:117] "RemoveContainer" containerID="6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec" Jul 23 16:22:09.337811 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.337796 2547 scope.go:117] "RemoveContainer" containerID="b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62" Jul 23 16:22:09.338051 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:22:09.338033 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62\": container with ID starting with b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62 not found: ID does not exist" containerID="b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62" Jul 23 16:22:09.338097 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.338061 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62"} err="failed to get container status \"b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62\": rpc error: code = NotFound desc = could not find container \"b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62\": container with ID starting with b8052594c71e4cb058b21d5e6b759f6f5e45bd9a3e8f92c17532286e9af09d62 not found: ID does not exist" Jul 23 16:22:09.338097 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.338078 2547 scope.go:117] "RemoveContainer" containerID="6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec" Jul 23 16:22:09.338302 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:22:09.338287 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec\": container with ID starting with 6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec not found: ID does not exist" containerID="6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec" Jul 23 16:22:09.338347 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.338305 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec"} err="failed to get container status \"6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec\": rpc error: code = NotFound desc = could not find container \"6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec\": container with ID starting with 6a1f35d44eaf31526b4ad3ada0339f96421c19ce55f03e257047aafbc743ddec not found: ID does not exist" Jul 23 16:22:09.343249 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.343230 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv"] Jul 23 16:22:09.347494 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:09.347473 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5bb1d507-openshift-default-549b9bf775-67jnv"] Jul 23 16:22:10.241284 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:22:10.241257 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" path="/var/lib/kubelet/pods/e28fd700-ddde-4f95-a78c-9f2086e0f9cc/volumes" Jul 23 16:23:47.255402 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255366 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt"] Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255750 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255762 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255779 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255785 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255802 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255808 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255861 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255869 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255876 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" Jul 23 16:23:47.255877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.255882 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="27eb7104-2da4-41eb-9b79-77c878868965" containerName="istio-proxy" Jul 23 16:23:47.258967 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.258948 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.261582 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.261562 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-77a41a7f-openshift-default-dockercfg-tw9tg\"" Jul 23 16:23:47.281147 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.281124 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt"] Jul 23 16:23:47.401742 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.401682 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-podinfo\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.401912 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.401755 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-token\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.401912 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.401782 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb9j7\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-kube-api-access-bb9j7\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.401912 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.401810 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-envoy\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.401912 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.401898 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-certs\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.402050 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.401938 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-socket\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.402050 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.401977 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8557a53-7943-48b8-a1ba-cab3794c839b-istiod-ca-cert\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.402114 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.402065 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-credential-socket\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.402114 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.402086 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-data\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503013 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.502974 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-certs\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503221 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503033 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-socket\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503221 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503068 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8557a53-7943-48b8-a1ba-cab3794c839b-istiod-ca-cert\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503221 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503146 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-credential-socket\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503221 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503172 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-data\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503221 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503211 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-podinfo\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503486 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503242 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-token\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503486 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503270 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb9j7\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-kube-api-access-bb9j7\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503486 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503311 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-envoy\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503486 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503444 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-certs\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503486 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503465 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-socket\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503763 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503671 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-credential-socket\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.503763 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.503724 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-data\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.504061 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.504038 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8557a53-7943-48b8-a1ba-cab3794c839b-istiod-ca-cert\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.505653 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.505600 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-envoy\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.505937 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.505917 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-podinfo\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.513104 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.513078 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-token\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.513211 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.513147 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb9j7\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-kube-api-access-bb9j7\") pod \"e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.569547 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.569521 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:47.723769 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.723739 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt"] Jul 23 16:23:47.725612 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:23:47.725586 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8557a53_7943_48b8_a1ba_cab3794c839b.slice/crio-a10c0ff324fe3a1a77df72b59610b8ce0613c1e8a134af7cc6241c3a8f9e23ae WatchSource:0}: Error finding container a10c0ff324fe3a1a77df72b59610b8ce0613c1e8a134af7cc6241c3a8f9e23ae: Status 404 returned error can't find the container with id a10c0ff324fe3a1a77df72b59610b8ce0613c1e8a134af7cc6241c3a8f9e23ae Jul 23 16:23:47.728143 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.728111 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:23:47.728229 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.728178 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:23:47.728229 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.728210 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:23:47.787170 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.787124 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" event={"ID":"b8557a53-7943-48b8-a1ba-cab3794c839b","Type":"ContainerStarted","Data":"012fa9827fc9c56b70e1269a1a12d455dd42db5ab0d8d6f812a5c45d0afdefc9"} Jul 23 16:23:47.787284 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.787202 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" event={"ID":"b8557a53-7943-48b8-a1ba-cab3794c839b","Type":"ContainerStarted","Data":"a10c0ff324fe3a1a77df72b59610b8ce0613c1e8a134af7cc6241c3a8f9e23ae"} Jul 23 16:23:47.843435 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:47.843381 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podStartSLOduration=0.843360071 podStartE2EDuration="843.360071ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:23:47.838510697 +0000 UTC m=+2014.115252110" watchObservedRunningTime="2026-07-23 16:23:47.843360071 +0000 UTC m=+2014.120101470" Jul 23 16:23:48.570375 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:48.570344 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:48.575056 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:48.575035 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:48.792042 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:48.792013 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:23:48.792986 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:23:48.792969 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:24:03.792973 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.792937 2547 patch_prober.go:28] interesting pod/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 23 16:24:03.793363 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.792995 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 23 16:24:03.872061 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.872032 2547 generic.go:356] "Generic (PLEG): container finished" podID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerID="012fa9827fc9c56b70e1269a1a12d455dd42db5ab0d8d6f812a5c45d0afdefc9" exitCode=0 Jul 23 16:24:03.872193 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.872059 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" event={"ID":"b8557a53-7943-48b8-a1ba-cab3794c839b","Type":"ContainerDied","Data":"012fa9827fc9c56b70e1269a1a12d455dd42db5ab0d8d6f812a5c45d0afdefc9"} Jul 23 16:24:03.872445 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.872429 2547 scope.go:117] "RemoveContainer" containerID="012fa9827fc9c56b70e1269a1a12d455dd42db5ab0d8d6f812a5c45d0afdefc9" Jul 23 16:24:03.873443 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.873418 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:24:03.873500 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.873482 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:24:03.873535 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:03.873514 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:24:04.881650 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:04.881617 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" event={"ID":"b8557a53-7943-48b8-a1ba-cab3794c839b","Type":"ContainerStarted","Data":"017a1f4800d9b2dffaf36a4e0d81ee83bff4e51ecec19570c67183337bab7f88"} Jul 23 16:24:05.570570 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:05.570533 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:24:05.570792 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:05.570760 2547 patch_prober.go:28] interesting pod/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 23 16:24:05.570849 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:05.570816 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 23 16:24:06.570852 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:06.570817 2547 patch_prober.go:28] interesting pod/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 23 16:24:06.571244 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:06.570876 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 23 16:24:07.572079 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:07.572050 2547 patch_prober.go:28] interesting pod/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 23 16:24:07.572501 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:07.572475 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 23 16:24:08.570337 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:08.570303 2547 patch_prober.go:28] interesting pod/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 23 16:24:08.570519 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:08.570368 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 23 16:24:09.573812 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:09.573784 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:24:09.574270 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:09.574151 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:24:09.574863 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:09.574844 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:24:23.764248 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:23.764218 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt"] Jul 23 16:24:23.764777 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:23.764436 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" containerID="cri-o://017a1f4800d9b2dffaf36a4e0d81ee83bff4e51ecec19570c67183337bab7f88" gracePeriod=30 Jul 23 16:24:24.576448 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:24.576416 2547 patch_prober.go:28] interesting pod/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 16:24:24.576615 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:24.576479 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 16:24:28.995655 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:28.995631 2547 generic.go:356] "Generic (PLEG): container finished" podID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerID="017a1f4800d9b2dffaf36a4e0d81ee83bff4e51ecec19570c67183337bab7f88" exitCode=0 Jul 23 16:24:28.995977 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:28.995712 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" event={"ID":"b8557a53-7943-48b8-a1ba-cab3794c839b","Type":"ContainerDied","Data":"017a1f4800d9b2dffaf36a4e0d81ee83bff4e51ecec19570c67183337bab7f88"} Jul 23 16:24:28.995977 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:28.995767 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" event={"ID":"b8557a53-7943-48b8-a1ba-cab3794c839b","Type":"ContainerDied","Data":"a10c0ff324fe3a1a77df72b59610b8ce0613c1e8a134af7cc6241c3a8f9e23ae"} Jul 23 16:24:28.995977 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:28.995785 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a10c0ff324fe3a1a77df72b59610b8ce0613c1e8a134af7cc6241c3a8f9e23ae" Jul 23 16:24:28.995977 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:28.995799 2547 scope.go:117] "RemoveContainer" containerID="012fa9827fc9c56b70e1269a1a12d455dd42db5ab0d8d6f812a5c45d0afdefc9" Jul 23 16:24:29.010564 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.010544 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:24:29.201184 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201155 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-socket\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201361 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201208 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-certs\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201361 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201251 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-token\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201361 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201275 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-envoy\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201361 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201312 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-credential-socket\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201361 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201358 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8557a53-7943-48b8-a1ba-cab3794c839b-istiod-ca-cert\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201628 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201387 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb9j7\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-kube-api-access-bb9j7\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201628 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201419 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-podinfo\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201628 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201459 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:24:29.201628 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201473 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-data\") pod \"b8557a53-7943-48b8-a1ba-cab3794c839b\" (UID: \"b8557a53-7943-48b8-a1ba-cab3794c839b\") " Jul 23 16:24:29.201628 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201573 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:24:29.201904 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201640 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:24:29.201904 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201882 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.202008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201893 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8557a53-7943-48b8-a1ba-cab3794c839b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:24:29.202008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201903 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.202008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.201957 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.202541 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.202514 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-data" (OuterVolumeSpecName: "istio-data") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:24:29.203523 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.203495 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:24:29.203612 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.203545 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-kube-api-access-bb9j7" (OuterVolumeSpecName: "kube-api-access-bb9j7") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "kube-api-access-bb9j7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:24:29.203612 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.203562 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-token" (OuterVolumeSpecName: "istio-token") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:24:29.203704 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.203623 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b8557a53-7943-48b8-a1ba-cab3794c839b" (UID: "b8557a53-7943-48b8-a1ba-cab3794c839b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:24:29.302637 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.302610 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.302637 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.302633 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.302637 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.302642 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8557a53-7943-48b8-a1ba-cab3794c839b-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.302851 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.302650 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb9j7\" (UniqueName: \"kubernetes.io/projected/b8557a53-7943-48b8-a1ba-cab3794c839b-kube-api-access-bb9j7\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.302851 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.302660 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:29.302851 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:29.302669 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8557a53-7943-48b8-a1ba-cab3794c839b-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:24:30.001350 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:30.001324 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt" Jul 23 16:24:30.105385 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:30.105358 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt"] Jul 23 16:24:30.127951 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:30.127918 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-77a41a7f-openshift-default-6bb76f5d7f-g7nlt"] Jul 23 16:24:30.240605 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:30.240578 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" path="/var/lib/kubelet/pods/b8557a53-7943-48b8-a1ba-cab3794c839b/volumes" Jul 23 16:24:43.667182 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667109 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj"] Jul 23 16:24:43.667532 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667473 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" Jul 23 16:24:43.667532 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667483 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" Jul 23 16:24:43.667532 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667499 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" Jul 23 16:24:43.667532 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667505 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" Jul 23 16:24:43.667532 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667512 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" Jul 23 16:24:43.667532 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667518 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28fd700-ddde-4f95-a78c-9f2086e0f9cc" containerName="istio-proxy" Jul 23 16:24:43.667773 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667578 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" Jul 23 16:24:43.667773 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.667585 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8557a53-7943-48b8-a1ba-cab3794c839b" containerName="istio-proxy" Jul 23 16:24:43.670342 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.670323 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.672683 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.672663 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5000bdcd-openshift-default-dockercfg-hv5zc\"" Jul 23 16:24:43.683888 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.683867 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj"] Jul 23 16:24:43.723869 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.723843 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-certs\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.723989 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.723873 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21346cc-e395-4015-905f-7551e1467a76-istio-podinfo\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.723989 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.723963 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21346cc-e395-4015-905f-7551e1467a76-istiod-ca-cert\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.724076 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.724037 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fmcb\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-kube-api-access-5fmcb\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.724108 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.724099 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-data\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.724146 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.724124 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-credential-socket\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.724146 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.724140 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-envoy\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.724224 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.724179 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-socket\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.724224 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.724204 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-istio-token\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.824809 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.824767 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-data\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.824809 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.824813 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-credential-socket\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825024 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.824840 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-envoy\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825024 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.824869 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-socket\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825024 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.824900 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-istio-token\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825024 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.824943 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-certs\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825024 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.824967 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21346cc-e395-4015-905f-7551e1467a76-istio-podinfo\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825285 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.825027 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21346cc-e395-4015-905f-7551e1467a76-istiod-ca-cert\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825285 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.825065 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fmcb\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-kube-api-access-5fmcb\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825285 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.825199 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-data\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825285 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.825233 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-credential-socket\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825497 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.825478 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-certs\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825602 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.825579 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-socket\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.825936 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.825914 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21346cc-e395-4015-905f-7551e1467a76-istiod-ca-cert\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.827157 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.827134 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-envoy\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.827608 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.827587 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21346cc-e395-4015-905f-7551e1467a76-istio-podinfo\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.835886 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.835867 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-istio-token\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.836044 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.836026 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fmcb\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-kube-api-access-5fmcb\") pod \"e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:43.980051 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:43.979971 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:44.112969 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:44.112949 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj"] Jul 23 16:24:44.115189 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:24:44.115167 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda21346cc_e395_4015_905f_7551e1467a76.slice/crio-47e2f2e3cce11aa5fb537dd123448dc219fec6acaf4bb9b71c79215896574391 WatchSource:0}: Error finding container 47e2f2e3cce11aa5fb537dd123448dc219fec6acaf4bb9b71c79215896574391: Status 404 returned error can't find the container with id 47e2f2e3cce11aa5fb537dd123448dc219fec6acaf4bb9b71c79215896574391 Jul 23 16:24:44.117316 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:44.117283 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:24:44.117409 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:44.117345 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:24:44.117409 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:44.117394 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:24:45.075556 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:45.075525 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" event={"ID":"a21346cc-e395-4015-905f-7551e1467a76","Type":"ContainerStarted","Data":"c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79"} Jul 23 16:24:45.075957 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:45.075602 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" event={"ID":"a21346cc-e395-4015-905f-7551e1467a76","Type":"ContainerStarted","Data":"47e2f2e3cce11aa5fb537dd123448dc219fec6acaf4bb9b71c79215896574391"} Jul 23 16:24:45.099446 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:45.099402 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podStartSLOduration=2.099387651 podStartE2EDuration="2.099387651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:24:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:24:45.097470652 +0000 UTC m=+2071.374212077" watchObservedRunningTime="2026-07-23 16:24:45.099387651 +0000 UTC m=+2071.376129050" Jul 23 16:24:45.980988 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:45.980957 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:45.985496 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:45.985475 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:46.080473 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:46.080446 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:24:46.081257 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:24:46.081241 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:25:02.102063 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.101467 2547 patch_prober.go:28] interesting pod/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 16:25:02.102063 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.101584 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 16:25:02.165123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.165097 2547 generic.go:356] "Generic (PLEG): container finished" podID="a21346cc-e395-4015-905f-7551e1467a76" containerID="c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79" exitCode=0 Jul 23 16:25:02.165233 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.165137 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" event={"ID":"a21346cc-e395-4015-905f-7551e1467a76","Type":"ContainerDied","Data":"c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79"} Jul 23 16:25:02.165570 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.165553 2547 scope.go:117] "RemoveContainer" containerID="c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79" Jul 23 16:25:02.166557 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.166527 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:25:02.166612 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.166600 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:25:02.166657 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:02.166631 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:25:03.172315 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:03.172280 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" event={"ID":"a21346cc-e395-4015-905f-7551e1467a76","Type":"ContainerStarted","Data":"5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0"} Jul 23 16:25:03.980959 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:03.980921 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:25:03.981166 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:03.981139 2547 patch_prober.go:28] interesting pod/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 16:25:03.981231 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:03.981208 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 16:25:04.981180 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:04.981141 2547 patch_prober.go:28] interesting pod/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 16:25:04.981564 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:04.981200 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 16:25:05.981253 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:05.981217 2547 patch_prober.go:28] interesting pod/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 16:25:05.981650 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:05.981273 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 16:25:06.980817 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:06.980774 2547 patch_prober.go:28] interesting pod/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 16:25:06.981004 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:06.980840 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 16:25:07.983918 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:07.983887 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:25:07.984293 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:07.984263 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:25:07.985038 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:07.985019 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:25:14.314118 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:14.314085 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:25:14.316483 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:14.316458 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:25:14.319474 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:14.319451 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:25:14.321757 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:25:14.321740 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:26:49.924241 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:49.924206 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj"] Jul 23 16:26:49.924854 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:49.924438 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" containerID="cri-o://5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0" gracePeriod=30 Jul 23 16:26:52.986642 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:52.986599 2547 patch_prober.go:28] interesting pod/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 16:26:52.987095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:52.986670 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 16:26:55.182732 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.182661 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:26:55.277463 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277429 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-data\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.277673 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277489 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fmcb\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-kube-api-access-5fmcb\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.277673 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277523 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-certs\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.277673 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277577 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-credential-socket\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.277890 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277723 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-istio-token\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.277890 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277787 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-envoy\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.277890 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277820 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-socket\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.277890 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277863 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21346cc-e395-4015-905f-7551e1467a76-istio-podinfo\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.278100 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277883 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:26:55.278100 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277898 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21346cc-e395-4015-905f-7551e1467a76-istiod-ca-cert\") pod \"a21346cc-e395-4015-905f-7551e1467a76\" (UID: \"a21346cc-e395-4015-905f-7551e1467a76\") " Jul 23 16:26:55.278100 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.277896 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:26:55.278253 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.278116 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:26:55.278309 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.278280 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.278309 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.278301 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.278409 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.278315 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.278409 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.278386 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a21346cc-e395-4015-905f-7551e1467a76-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:26:55.278494 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.278471 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-data" (OuterVolumeSpecName: "istio-data") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:26:55.280352 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.280326 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:26:55.280418 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.280352 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-istio-token" (OuterVolumeSpecName: "istio-token") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:26:55.280418 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.280349 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-kube-api-access-5fmcb" (OuterVolumeSpecName: "kube-api-access-5fmcb") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "kube-api-access-5fmcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:26:55.280489 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.280419 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a21346cc-e395-4015-905f-7551e1467a76-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a21346cc-e395-4015-905f-7551e1467a76" (UID: "a21346cc-e395-4015-905f-7551e1467a76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:26:55.379589 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.379551 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.379589 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.379578 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fmcb\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-kube-api-access-5fmcb\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.379589 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.379587 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a21346cc-e395-4015-905f-7551e1467a76-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.379589 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.379596 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a21346cc-e395-4015-905f-7551e1467a76-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.379954 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.379605 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a21346cc-e395-4015-905f-7551e1467a76-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.379954 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.379612 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a21346cc-e395-4015-905f-7551e1467a76-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:26:55.738794 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.738759 2547 generic.go:356] "Generic (PLEG): container finished" podID="a21346cc-e395-4015-905f-7551e1467a76" containerID="5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0" exitCode=0 Jul 23 16:26:55.738794 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.738798 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" event={"ID":"a21346cc-e395-4015-905f-7551e1467a76","Type":"ContainerDied","Data":"5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0"} Jul 23 16:26:55.738999 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.738809 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" Jul 23 16:26:55.738999 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.738841 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj" event={"ID":"a21346cc-e395-4015-905f-7551e1467a76","Type":"ContainerDied","Data":"47e2f2e3cce11aa5fb537dd123448dc219fec6acaf4bb9b71c79215896574391"} Jul 23 16:26:55.738999 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.738862 2547 scope.go:117] "RemoveContainer" containerID="5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0" Jul 23 16:26:55.747702 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.747670 2547 scope.go:117] "RemoveContainer" containerID="c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79" Jul 23 16:26:55.755278 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.755261 2547 scope.go:117] "RemoveContainer" containerID="5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0" Jul 23 16:26:55.755532 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:26:55.755514 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0\": container with ID starting with 5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0 not found: ID does not exist" containerID="5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0" Jul 23 16:26:55.755580 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.755540 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0"} err="failed to get container status \"5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0\": rpc error: code = NotFound desc = could not find container \"5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0\": container with ID starting with 5f6e76a08ab7fd0f1659f219621a82f654ed064e712699fcc82dc2ab2fb560c0 not found: ID does not exist" Jul 23 16:26:55.755580 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.755558 2547 scope.go:117] "RemoveContainer" containerID="c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79" Jul 23 16:26:55.755791 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:26:55.755773 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79\": container with ID starting with c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79 not found: ID does not exist" containerID="c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79" Jul 23 16:26:55.755853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.755799 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79"} err="failed to get container status \"c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79\": rpc error: code = NotFound desc = could not find container \"c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79\": container with ID starting with c1c5488534b0308dc0ba23fed16579c8d0ac9693618c7f10d9a09212eb87df79 not found: ID does not exist" Jul 23 16:26:55.761154 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.761132 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj"] Jul 23 16:26:55.765249 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:55.765231 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5000bdcd-openshift-default-8565ff875f-rv8wj"] Jul 23 16:26:56.241089 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:26:56.241052 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a21346cc-e395-4015-905f-7551e1467a76" path="/var/lib/kubelet/pods/a21346cc-e395-4015-905f-7551e1467a76/volumes" Jul 23 16:27:46.350726 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.350636 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv"] Jul 23 16:27:46.351191 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.351037 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" Jul 23 16:27:46.351191 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.351049 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" Jul 23 16:27:46.351191 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.351058 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" Jul 23 16:27:46.351191 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.351063 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" Jul 23 16:27:46.351191 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.351124 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" Jul 23 16:27:46.354328 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.354312 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.356711 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.356683 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-652934-openshift-default-dockercfg-q6bz7\"" Jul 23 16:27:46.366345 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.366323 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv"] Jul 23 16:27:46.439052 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.439019 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-certs\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.439365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.439341 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-socket\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.439600 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.439576 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istiod-ca-cert\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.439803 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.439786 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-token\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.440005 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.439983 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-data\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.440143 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.440014 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-podinfo\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.440143 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.440060 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-credential-socket\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.440143 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.440085 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h82w8\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-kube-api-access-h82w8\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.440143 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.440109 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-envoy\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541179 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541144 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istiod-ca-cert\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541199 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-token\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541225 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-data\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541242 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-podinfo\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541268 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-credential-socket\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541286 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h82w8\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-kube-api-access-h82w8\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541302 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-envoy\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541351 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-certs\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541843 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541375 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-socket\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541843 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541648 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-data\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541843 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541745 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-credential-socket\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.541843 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541812 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-certs\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.542041 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.541847 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-socket\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.542041 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.542006 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istiod-ca-cert\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.543670 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.543644 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-envoy\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.543896 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.543875 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-podinfo\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.549420 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.549396 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-token\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.550102 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.550077 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h82w8\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-kube-api-access-h82w8\") pod \"e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.664128 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.664047 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:46.809067 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.809044 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv"] Jul 23 16:27:46.811226 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:27:46.811197 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82a6b0d9_8aa2_488b_9401_011ca8bc0b00.slice/crio-55153fd2a37e215e3c6437cd622a99823a8bafaad81665de31a430a51a14ec5b WatchSource:0}: Error finding container 55153fd2a37e215e3c6437cd622a99823a8bafaad81665de31a430a51a14ec5b: Status 404 returned error can't find the container with id 55153fd2a37e215e3c6437cd622a99823a8bafaad81665de31a430a51a14ec5b Jul 23 16:27:46.813756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.813721 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:27:46.813847 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.813802 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:27:46.813891 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.813846 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:27:46.995329 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.995239 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" event={"ID":"82a6b0d9-8aa2-488b-9401-011ca8bc0b00","Type":"ContainerStarted","Data":"9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3"} Jul 23 16:27:46.995329 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:46.995304 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" event={"ID":"82a6b0d9-8aa2-488b-9401-011ca8bc0b00","Type":"ContainerStarted","Data":"55153fd2a37e215e3c6437cd622a99823a8bafaad81665de31a430a51a14ec5b"} Jul 23 16:27:47.018013 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:47.017964 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podStartSLOduration=1.017949862 podStartE2EDuration="1.017949862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:27:47.016134837 +0000 UTC m=+2253.292876239" watchObservedRunningTime="2026-07-23 16:27:47.017949862 +0000 UTC m=+2253.294691320" Jul 23 16:27:47.664993 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:47.664963 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:47.669788 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:47.669765 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:48.000428 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:48.000348 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:48.001255 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:48.001236 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:27:54.159625 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.159596 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt"] Jul 23 16:27:54.160116 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.160046 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="a21346cc-e395-4015-905f-7551e1467a76" containerName="istio-proxy" Jul 23 16:27:54.163146 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.163128 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.166137 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.166112 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d1deea-openshift-default-dockercfg-6f6nr\"" Jul 23 16:27:54.179144 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.179118 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt"] Jul 23 16:27:54.311445 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311413 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-socket\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311625 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311457 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-token\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311625 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311528 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-podinfo\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311625 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311554 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-certs\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311625 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311579 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-data\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311829 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311628 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-envoy\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311829 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311644 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-credential-socket\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311829 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311673 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ad1a9e-1471-4b17-93dd-c6e38492e853-istiod-ca-cert\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.311829 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.311751 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jglw\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-kube-api-access-8jglw\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412447 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412365 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-data\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412447 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412417 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-envoy\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412447 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412439 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-credential-socket\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412463 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ad1a9e-1471-4b17-93dd-c6e38492e853-istiod-ca-cert\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412521 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jglw\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-kube-api-access-8jglw\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412583 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-socket\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412615 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-token\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412650 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-podinfo\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.412756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412687 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-certs\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.413083 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412786 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-data\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.413083 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412800 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-credential-socket\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.413083 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.412977 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-socket\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.413252 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.413203 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-certs\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.413423 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.413401 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ad1a9e-1471-4b17-93dd-c6e38492e853-istiod-ca-cert\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.414656 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.414639 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-envoy\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.414978 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.414961 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-podinfo\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.420757 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.420730 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-token\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.420904 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.420889 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jglw\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-kube-api-access-8jglw\") pod \"e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.474591 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.474558 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:54.611680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.611657 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt"] Jul 23 16:27:54.613623 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:27:54.613592 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ad1a9e_1471_4b17_93dd_c6e38492e853.slice/crio-97e58edfa1cba5bc49646e6fd60336560e34ee9d02b5ea488794d322be980859 WatchSource:0}: Error finding container 97e58edfa1cba5bc49646e6fd60336560e34ee9d02b5ea488794d322be980859: Status 404 returned error can't find the container with id 97e58edfa1cba5bc49646e6fd60336560e34ee9d02b5ea488794d322be980859 Jul 23 16:27:54.618652 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.618611 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:27:54.618795 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.618711 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:27:54.618795 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:54.618753 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:27:55.033512 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:55.033427 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" event={"ID":"51ad1a9e-1471-4b17-93dd-c6e38492e853","Type":"ContainerStarted","Data":"ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419"} Jul 23 16:27:55.033512 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:55.033479 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" event={"ID":"51ad1a9e-1471-4b17-93dd-c6e38492e853","Type":"ContainerStarted","Data":"97e58edfa1cba5bc49646e6fd60336560e34ee9d02b5ea488794d322be980859"} Jul 23 16:27:55.053707 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:55.053648 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podStartSLOduration=1.053634295 podStartE2EDuration="1.053634295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:27:55.052762697 +0000 UTC m=+2261.329504092" watchObservedRunningTime="2026-07-23 16:27:55.053634295 +0000 UTC m=+2261.330375697" Jul 23 16:27:55.475056 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:55.475017 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:55.479823 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:55.479799 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:56.038834 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:56.038810 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:27:56.039624 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:27:56.039608 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:28:04.041891 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:04.041847 2547 patch_prober.go:28] interesting pod/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 16:28:04.042300 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:04.041937 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 16:28:06.092734 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:06.092615 2547 generic.go:356] "Generic (PLEG): container finished" podID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerID="9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3" exitCode=0 Jul 23 16:28:06.092734 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:06.092672 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" event={"ID":"82a6b0d9-8aa2-488b-9401-011ca8bc0b00","Type":"ContainerDied","Data":"9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3"} Jul 23 16:28:06.093330 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:06.093238 2547 scope.go:117] "RemoveContainer" containerID="9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3" Jul 23 16:28:06.094306 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:06.094275 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:28:06.094396 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:06.094342 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:28:06.094396 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:06.094375 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:28:07.099225 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:07.099191 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" event={"ID":"82a6b0d9-8aa2-488b-9401-011ca8bc0b00","Type":"ContainerStarted","Data":"0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776"} Jul 23 16:28:07.665168 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:07.665127 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:28:07.665381 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:07.665256 2547 patch_prober.go:28] interesting pod/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 16:28:07.665381 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:07.665300 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 16:28:08.664876 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:08.664832 2547 patch_prober.go:28] interesting pod/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 16:28:08.665367 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:08.664901 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 16:28:09.665133 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:09.665097 2547 patch_prober.go:28] interesting pod/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 16:28:09.665734 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:09.665685 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 16:28:10.665011 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:10.664969 2547 patch_prober.go:28] interesting pod/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 16:28:10.665208 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:10.665032 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 16:28:11.667967 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:11.667928 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:28:11.668439 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:11.668323 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:28:11.668956 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:11.668938 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:28:12.080873 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:12.080835 2547 patch_prober.go:28] interesting pod/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 16:28:12.081034 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:12.080905 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 16:28:14.137536 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:14.137507 2547 generic.go:356] "Generic (PLEG): container finished" podID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerID="ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419" exitCode=0 Jul 23 16:28:14.137975 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:14.137578 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" event={"ID":"51ad1a9e-1471-4b17-93dd-c6e38492e853","Type":"ContainerDied","Data":"ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419"} Jul 23 16:28:14.138053 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:14.138036 2547 scope.go:117] "RemoveContainer" containerID="ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419" Jul 23 16:28:14.139029 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:14.139000 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:28:14.139123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:14.139065 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:28:14.139123 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:14.139093 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:28:15.148405 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:15.148366 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" event={"ID":"51ad1a9e-1471-4b17-93dd-c6e38492e853","Type":"ContainerStarted","Data":"67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8"} Jul 23 16:28:15.475196 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:15.475105 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:28:15.475368 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:15.475347 2547 patch_prober.go:28] interesting pod/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 16:28:15.475425 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:15.475407 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 16:28:16.475358 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:16.475326 2547 patch_prober.go:28] interesting pod/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 16:28:16.475779 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:16.475383 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 16:28:17.475421 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:17.475383 2547 patch_prober.go:28] interesting pod/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 16:28:17.475943 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:17.475442 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 16:28:18.475879 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:18.475841 2547 patch_prober.go:28] interesting pod/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 16:28:18.476274 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:18.475895 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 16:28:19.478629 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:19.478601 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:28:19.479054 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:19.478950 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:28:19.479751 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:28:19.479734 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:29:54.342979 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:54.342949 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv"] Jul 23 16:29:54.343543 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:54.343144 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" containerID="cri-o://0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776" gracePeriod=30 Jul 23 16:29:56.671047 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:56.671016 2547 patch_prober.go:28] interesting pod/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 16:29:56.671426 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:56.671080 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 16:29:59.592596 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.592573 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:29:59.679556 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.679469 2547 generic.go:356] "Generic (PLEG): container finished" podID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerID="0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776" exitCode=0 Jul 23 16:29:59.679556 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.679524 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" Jul 23 16:29:59.679556 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.679547 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" event={"ID":"82a6b0d9-8aa2-488b-9401-011ca8bc0b00","Type":"ContainerDied","Data":"0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776"} Jul 23 16:29:59.679869 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.679600 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv" event={"ID":"82a6b0d9-8aa2-488b-9401-011ca8bc0b00","Type":"ContainerDied","Data":"55153fd2a37e215e3c6437cd622a99823a8bafaad81665de31a430a51a14ec5b"} Jul 23 16:29:59.679869 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.679623 2547 scope.go:117] "RemoveContainer" containerID="0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776" Jul 23 16:29:59.688527 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.688512 2547 scope.go:117] "RemoveContainer" containerID="9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3" Jul 23 16:29:59.695991 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.695975 2547 scope.go:117] "RemoveContainer" containerID="0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776" Jul 23 16:29:59.696223 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:29:59.696206 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776\": container with ID starting with 0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776 not found: ID does not exist" containerID="0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776" Jul 23 16:29:59.696264 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.696233 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776"} err="failed to get container status \"0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776\": rpc error: code = NotFound desc = could not find container \"0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776\": container with ID starting with 0dc0db85e2277b0e5d4117d493fed77713d63eba32b857992df8f325580a7776 not found: ID does not exist" Jul 23 16:29:59.696264 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.696250 2547 scope.go:117] "RemoveContainer" containerID="9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3" Jul 23 16:29:59.696492 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:29:59.696475 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3\": container with ID starting with 9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3 not found: ID does not exist" containerID="9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3" Jul 23 16:29:59.696537 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.696500 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3"} err="failed to get container status \"9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3\": rpc error: code = NotFound desc = could not find container \"9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3\": container with ID starting with 9c9aebd4ececc0f816030c5615edd20d39418079fb0874e198b86f2d781544f3 not found: ID does not exist" Jul 23 16:29:59.713902 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.713882 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h82w8\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-kube-api-access-h82w8\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.713983 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.713921 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-data\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.713983 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.713955 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-token\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.714073 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.713985 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-socket\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.714073 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714000 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-credential-socket\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.714073 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714020 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istiod-ca-cert\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.714238 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714078 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-envoy\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.714238 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714166 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-podinfo\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.714238 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714194 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-certs\") pod \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\" (UID: \"82a6b0d9-8aa2-488b-9401-011ca8bc0b00\") " Jul 23 16:29:59.714391 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714263 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:29:59.714391 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714300 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:29:59.714531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714511 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:29:59.714590 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714567 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.714644 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714591 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.714748 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714724 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:29:59.714965 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.714939 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-data" (OuterVolumeSpecName: "istio-data") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:29:59.716153 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.716130 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-kube-api-access-h82w8" (OuterVolumeSpecName: "kube-api-access-h82w8") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "kube-api-access-h82w8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:29:59.716255 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.716147 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-token" (OuterVolumeSpecName: "istio-token") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:29:59.716305 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.716277 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:29:59.716346 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.716314 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82a6b0d9-8aa2-488b-9401-011ca8bc0b00" (UID: "82a6b0d9-8aa2-488b-9401-011ca8bc0b00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:29:59.815948 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.815915 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.815948 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.815946 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.815948 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.815956 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.816149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.815964 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.816149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.815974 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h82w8\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-kube-api-access-h82w8\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.816149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.815982 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:29:59.816149 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:29:59.815990 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82a6b0d9-8aa2-488b-9401-011ca8bc0b00-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:00.003004 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:00.002980 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv"] Jul 23 16:30:00.008272 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:00.008245 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-652934-openshift-default-78bb7ff8dc-dhtwv"] Jul 23 16:30:00.241935 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:00.241855 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" path="/var/lib/kubelet/pods/82a6b0d9-8aa2-488b-9401-011ca8bc0b00/volumes" Jul 23 16:30:14.346183 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:14.346158 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:30:14.348561 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:14.348537 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:30:14.353393 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:14.353371 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:30:14.355418 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:14.355392 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:30:14.377666 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:14.377633 2547 scope.go:117] "RemoveContainer" containerID="017a1f4800d9b2dffaf36a4e0d81ee83bff4e51ecec19570c67183337bab7f88" Jul 23 16:30:16.265311 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:16.265278 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt"] Jul 23 16:30:16.265709 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:16.265491 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" containerID="cri-o://67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8" gracePeriod=30 Jul 23 16:30:17.197444 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.197412 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74"] Jul 23 16:30:17.197850 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.197833 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" Jul 23 16:30:17.197925 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.197855 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" Jul 23 16:30:17.197925 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.197882 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" Jul 23 16:30:17.197925 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.197891 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" Jul 23 16:30:17.198063 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.198037 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" Jul 23 16:30:17.198063 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.198048 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="82a6b0d9-8aa2-488b-9401-011ca8bc0b00" containerName="istio-proxy" Jul 23 16:30:17.203255 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.203206 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.206230 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.206212 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-dbc842-openshift-default-dockercfg-c9bkg\"" Jul 23 16:30:17.215225 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.215202 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74"] Jul 23 16:30:17.280080 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280046 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-credential-socket\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280101 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-istio-token\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280164 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-data\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280201 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11192947-adab-4eb0-9780-a81e7f8e6356-istiod-ca-cert\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280219 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t7hw\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-kube-api-access-5t7hw\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280248 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-envoy\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280290 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-certs\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280329 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-socket\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.280429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.280389 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11192947-adab-4eb0-9780-a81e7f8e6356-istio-podinfo\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.381111 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.381074 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11192947-adab-4eb0-9780-a81e7f8e6356-istio-podinfo\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.381904 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.381869 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-credential-socket\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382177 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382152 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-istio-token\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382369 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382334 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-data\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382461 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382372 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11192947-adab-4eb0-9780-a81e7f8e6356-istiod-ca-cert\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382461 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382402 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-credential-socket\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382461 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382398 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t7hw\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-kube-api-access-5t7hw\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382461 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382446 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-envoy\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382677 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382508 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-certs\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382775 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382754 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-data\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.382950 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382931 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-socket\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.383132 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.383104 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11192947-adab-4eb0-9780-a81e7f8e6356-istiod-ca-cert\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.383280 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.382961 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-certs\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.383391 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.383166 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-socket\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.384436 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.384398 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11192947-adab-4eb0-9780-a81e7f8e6356-istio-podinfo\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.385219 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.385194 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-envoy\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.392478 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.392422 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t7hw\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-kube-api-access-5t7hw\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.393038 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.393007 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-istio-token\") pod \"e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.515101 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.515009 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:17.658596 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.658484 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74"] Jul 23 16:30:17.661708 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:30:17.661663 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11192947_adab_4eb0_9780_a81e7f8e6356.slice/crio-ff8a31c6043db9a3b227428f851a71da96cc16268f03c73efcfa4021efcc79f9 WatchSource:0}: Error finding container ff8a31c6043db9a3b227428f851a71da96cc16268f03c73efcfa4021efcc79f9: Status 404 returned error can't find the container with id ff8a31c6043db9a3b227428f851a71da96cc16268f03c73efcfa4021efcc79f9 Jul 23 16:30:17.664357 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.664314 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:17.664475 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.664408 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:17.664475 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.664452 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:17.770353 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.770258 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" event={"ID":"11192947-adab-4eb0-9780-a81e7f8e6356","Type":"ContainerStarted","Data":"2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12"} Jul 23 16:30:17.770353 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.770315 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" event={"ID":"11192947-adab-4eb0-9780-a81e7f8e6356","Type":"ContainerStarted","Data":"ff8a31c6043db9a3b227428f851a71da96cc16268f03c73efcfa4021efcc79f9"} Jul 23 16:30:17.792808 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:17.792754 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" podStartSLOduration=0.792738421 podStartE2EDuration="792.738421ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:30:17.790182169 +0000 UTC m=+2404.066934357" watchObservedRunningTime="2026-07-23 16:30:17.792738421 +0000 UTC m=+2404.069479822" Jul 23 16:30:18.516009 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:18.515972 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:18.520940 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:18.520913 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:18.775401 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:18.775318 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:18.776380 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:18.776358 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:19.481240 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:19.481206 2547 patch_prober.go:28] interesting pod/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 16:30:19.481420 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:19.481258 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 16:30:21.521215 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.521193 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:30:21.630063 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630031 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-socket\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630249 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630088 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-credential-socket\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630249 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630144 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-data\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630249 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630169 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-certs\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630249 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630199 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-podinfo\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630249 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630240 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-envoy\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630477 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630276 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-token\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630477 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630307 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ad1a9e-1471-4b17-93dd-c6e38492e853-istiod-ca-cert\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630477 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630363 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jglw\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-kube-api-access-8jglw\") pod \"51ad1a9e-1471-4b17-93dd-c6e38492e853\" (UID: \"51ad1a9e-1471-4b17-93dd-c6e38492e853\") " Jul 23 16:30:21.630477 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.630337 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:30:21.632962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.631071 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:30:21.632962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.631162 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-data" (OuterVolumeSpecName: "istio-data") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:30:21.632962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.631319 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:30:21.632962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.631343 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.632962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.631380 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.632962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.631394 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.632962 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.631743 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51ad1a9e-1471-4b17-93dd-c6e38492e853-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:30:21.633393 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.633369 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-kube-api-access-8jglw" (OuterVolumeSpecName: "kube-api-access-8jglw") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "kube-api-access-8jglw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:30:21.638178 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.633657 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-token" (OuterVolumeSpecName: "istio-token") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:30:21.638178 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.633734 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:30:21.639599 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.639565 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51ad1a9e-1471-4b17-93dd-c6e38492e853" (UID: "51ad1a9e-1471-4b17-93dd-c6e38492e853"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:30:21.732439 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.732404 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.732439 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.732436 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.732439 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.732447 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.732717 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.732455 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.732717 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.732463 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51ad1a9e-1471-4b17-93dd-c6e38492e853-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.732717 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.732471 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jglw\" (UniqueName: \"kubernetes.io/projected/51ad1a9e-1471-4b17-93dd-c6e38492e853-kube-api-access-8jglw\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:30:21.791474 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.791433 2547 generic.go:356] "Generic (PLEG): container finished" podID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerID="67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8" exitCode=0 Jul 23 16:30:21.791638 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.791518 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" Jul 23 16:30:21.791638 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.791514 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" event={"ID":"51ad1a9e-1471-4b17-93dd-c6e38492e853","Type":"ContainerDied","Data":"67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8"} Jul 23 16:30:21.791638 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.791600 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt" event={"ID":"51ad1a9e-1471-4b17-93dd-c6e38492e853","Type":"ContainerDied","Data":"97e58edfa1cba5bc49646e6fd60336560e34ee9d02b5ea488794d322be980859"} Jul 23 16:30:21.791638 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.791623 2547 scope.go:117] "RemoveContainer" containerID="67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8" Jul 23 16:30:21.800936 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.800914 2547 scope.go:117] "RemoveContainer" containerID="ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419" Jul 23 16:30:21.809429 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.809409 2547 scope.go:117] "RemoveContainer" containerID="67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8" Jul 23 16:30:21.809717 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:30:21.809684 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8\": container with ID starting with 67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8 not found: ID does not exist" containerID="67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8" Jul 23 16:30:21.809792 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.809728 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8"} err="failed to get container status \"67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8\": rpc error: code = NotFound desc = could not find container \"67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8\": container with ID starting with 67588114cae681374dfd5684c79a472f68ae52f9752646f5e12ae102d97361a8 not found: ID does not exist" Jul 23 16:30:21.809792 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.809747 2547 scope.go:117] "RemoveContainer" containerID="ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419" Jul 23 16:30:21.810002 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:30:21.809984 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419\": container with ID starting with ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419 not found: ID does not exist" containerID="ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419" Jul 23 16:30:21.810040 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.810009 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419"} err="failed to get container status \"ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419\": rpc error: code = NotFound desc = could not find container \"ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419\": container with ID starting with ed23d51a5d6ccd1a396ef834b5356121844ad813f174da0942bb26ff1483a419 not found: ID does not exist" Jul 23 16:30:21.814414 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.814391 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt"] Jul 23 16:30:21.818459 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:21.818436 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d1deea-openshift-default-754fcb9555-dhclt"] Jul 23 16:30:22.240910 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:22.240878 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" path="/var/lib/kubelet/pods/51ad1a9e-1471-4b17-93dd-c6e38492e853/volumes" Jul 23 16:30:25.113353 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.113323 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r"] Jul 23 16:30:25.113909 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.113891 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" Jul 23 16:30:25.113989 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.113912 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" Jul 23 16:30:25.113989 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.113945 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" Jul 23 16:30:25.113989 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.113954 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" Jul 23 16:30:25.114137 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.114059 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" Jul 23 16:30:25.114137 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.114073 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ad1a9e-1471-4b17-93dd-c6e38492e853" containerName="istio-proxy" Jul 23 16:30:25.118953 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.118924 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.121869 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.121844 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6fc9aa-openshift-default-dockercfg-d7f57\"" Jul 23 16:30:25.129395 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.129362 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r"] Jul 23 16:30:25.162832 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.162801 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-certs\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.162971 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.162843 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-envoy\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.162971 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.162911 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0170e9f0-4e12-4769-b016-a65450e971a7-istio-podinfo\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.162971 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.162930 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-istio-token\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.163095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.162978 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-credential-socket\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.163095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.163016 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-socket\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.163095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.163038 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-data\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.163095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.163072 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h59c8\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-kube-api-access-h59c8\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.163095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.163091 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0170e9f0-4e12-4769-b016-a65450e971a7-istiod-ca-cert\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264379 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264344 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-socket\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264576 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264386 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-data\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264576 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264428 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h59c8\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-kube-api-access-h59c8\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264576 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264455 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0170e9f0-4e12-4769-b016-a65450e971a7-istiod-ca-cert\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264576 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264520 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-certs\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264823 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264709 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-envoy\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264817 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0170e9f0-4e12-4769-b016-a65450e971a7-istio-podinfo\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264822 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-socket\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.264881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264857 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-istio-token\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.265027 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264900 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-credential-socket\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.265027 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264857 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-data\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.265027 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.264958 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-certs\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.265231 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.265209 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-credential-socket\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.265231 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.265227 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0170e9f0-4e12-4769-b016-a65450e971a7-istiod-ca-cert\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.266848 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.266831 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-envoy\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.267200 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.267185 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0170e9f0-4e12-4769-b016-a65450e971a7-istio-podinfo\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.275717 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.273289 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-istio-token\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.275717 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.273341 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h59c8\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-kube-api-access-h59c8\") pod \"e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.432296 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.432197 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:25.574423 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.574391 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r"] Jul 23 16:30:25.576010 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:30:25.575979 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0170e9f0_4e12_4769_b016_a65450e971a7.slice/crio-cca44beb016c82bfca3465f863467344c5384f2d4ebec99fde8db06a7b718659 WatchSource:0}: Error finding container cca44beb016c82bfca3465f863467344c5384f2d4ebec99fde8db06a7b718659: Status 404 returned error can't find the container with id cca44beb016c82bfca3465f863467344c5384f2d4ebec99fde8db06a7b718659 Jul 23 16:30:25.577975 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.577942 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:25.578060 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.578006 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:25.578060 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.578034 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:25.815318 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.815284 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" event={"ID":"0170e9f0-4e12-4769-b016-a65450e971a7","Type":"ContainerStarted","Data":"6029de3ccabc0746526b59f8903d580cc9668a2134db12fd210a315fa5777172"} Jul 23 16:30:25.815526 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.815349 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" event={"ID":"0170e9f0-4e12-4769-b016-a65450e971a7","Type":"ContainerStarted","Data":"cca44beb016c82bfca3465f863467344c5384f2d4ebec99fde8db06a7b718659"} Jul 23 16:30:25.833710 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:25.833623 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podStartSLOduration=0.833602732 podStartE2EDuration="833.602732ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:30:25.833249474 +0000 UTC m=+2412.109990897" watchObservedRunningTime="2026-07-23 16:30:25.833602732 +0000 UTC m=+2412.110344133" Jul 23 16:30:26.433280 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:26.433244 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:26.437967 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:26.437944 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:26.820095 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:26.820072 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:26.820915 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:26.820900 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:34.818862 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:34.818822 2547 patch_prober.go:28] interesting pod/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 16:30:34.819287 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:34.818888 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 16:30:36.881387 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:36.881350 2547 generic.go:356] "Generic (PLEG): container finished" podID="11192947-adab-4eb0-9780-a81e7f8e6356" containerID="2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12" exitCode=0 Jul 23 16:30:36.881850 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:36.881396 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" event={"ID":"11192947-adab-4eb0-9780-a81e7f8e6356","Type":"ContainerDied","Data":"2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12"} Jul 23 16:30:36.881909 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:36.881881 2547 scope.go:117] "RemoveContainer" containerID="2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12" Jul 23 16:30:36.882994 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:36.882965 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:36.883064 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:36.883033 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:36.883102 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:36.883064 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:37.889043 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:37.889009 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" event={"ID":"11192947-adab-4eb0-9780-a81e7f8e6356","Type":"ContainerStarted","Data":"ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a"} Jul 23 16:30:38.515885 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:38.515852 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:38.516093 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:38.515966 2547 patch_prober.go:28] interesting pod/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 16:30:38.516093 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:38.516003 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 16:30:39.516054 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:39.516018 2547 patch_prober.go:28] interesting pod/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 16:30:39.516442 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:39.516077 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 16:30:40.516376 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:40.516288 2547 patch_prober.go:28] interesting pod/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 16:30:40.516376 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:40.516345 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 16:30:41.515593 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:41.515556 2547 patch_prober.go:28] interesting pod/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 16:30:41.515836 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:41.515616 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 16:30:42.519248 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:42.519208 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:42.519756 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:42.519595 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:42.520398 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:42.520382 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:30:42.860848 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:42.860812 2547 patch_prober.go:28] interesting pod/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 16:30:42.861028 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:42.860869 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 16:30:43.923728 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:43.923672 2547 generic.go:356] "Generic (PLEG): container finished" podID="0170e9f0-4e12-4769-b016-a65450e971a7" containerID="6029de3ccabc0746526b59f8903d580cc9668a2134db12fd210a315fa5777172" exitCode=0 Jul 23 16:30:43.924220 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:43.923745 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" event={"ID":"0170e9f0-4e12-4769-b016-a65450e971a7","Type":"ContainerDied","Data":"6029de3ccabc0746526b59f8903d580cc9668a2134db12fd210a315fa5777172"} Jul 23 16:30:43.924348 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:43.924330 2547 scope.go:117] "RemoveContainer" containerID="6029de3ccabc0746526b59f8903d580cc9668a2134db12fd210a315fa5777172" Jul 23 16:30:43.925306 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:43.925277 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:43.925400 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:43.925348 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:43.925400 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:43.925379 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:30:44.930676 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:44.930637 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" event={"ID":"0170e9f0-4e12-4769-b016-a65450e971a7","Type":"ContainerStarted","Data":"e1fe1d9043dc50b00345ebe50b5ca38b8fa07e698ef77882e30b41b957604904"} Jul 23 16:30:45.433481 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:45.433437 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:45.433879 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:45.433851 2547 patch_prober.go:28] interesting pod/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 16:30:45.434047 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:45.434022 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 16:30:46.433681 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:46.433647 2547 patch_prober.go:28] interesting pod/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 16:30:46.434291 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:46.434256 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 16:30:47.433669 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:47.433618 2547 patch_prober.go:28] interesting pod/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 16:30:47.434004 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:47.433966 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 16:30:48.433055 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:48.433021 2547 patch_prober.go:28] interesting pod/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 16:30:48.433322 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:48.433290 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 16:30:49.435857 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:49.435822 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:49.436329 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:49.436106 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:30:49.436713 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:30:49.436675 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:31:07.126146 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:07.126110 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74"] Jul 23 16:31:07.126626 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:07.126335 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" containerID="cri-o://ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a" gracePeriod=30 Jul 23 16:31:12.381939 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.381914 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:31:12.451635 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451552 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11192947-adab-4eb0-9780-a81e7f8e6356-istio-podinfo\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.451635 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451599 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-credential-socket\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.451881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451642 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-socket\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.451881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451663 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-data\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.451881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451681 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-certs\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.451881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451744 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11192947-adab-4eb0-9780-a81e7f8e6356-istiod-ca-cert\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.451881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451780 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-istio-token\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.451881 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451826 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t7hw\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-kube-api-access-5t7hw\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.452213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451906 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-envoy\") pod \"11192947-adab-4eb0-9780-a81e7f8e6356\" (UID: \"11192947-adab-4eb0-9780-a81e7f8e6356\") " Jul 23 16:31:12.452213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451969 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:12.452213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.451988 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:12.452213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.452088 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:12.452419 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.452272 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.452419 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.452295 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.452419 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.452310 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.452564 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.452499 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11192947-adab-4eb0-9780-a81e7f8e6356-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:31:12.452740 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.452683 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-data" (OuterVolumeSpecName: "istio-data") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:12.454387 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.454121 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-istio-token" (OuterVolumeSpecName: "istio-token") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:31:12.454387 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.454164 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11192947-adab-4eb0-9780-a81e7f8e6356-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:31:12.454568 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.454433 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-kube-api-access-5t7hw" (OuterVolumeSpecName: "kube-api-access-5t7hw") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "kube-api-access-5t7hw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:31:12.454568 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.454514 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11192947-adab-4eb0-9780-a81e7f8e6356" (UID: "11192947-adab-4eb0-9780-a81e7f8e6356"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:12.553573 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.553526 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.553573 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.553573 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11192947-adab-4eb0-9780-a81e7f8e6356-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.553573 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.553583 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11192947-adab-4eb0-9780-a81e7f8e6356-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.553573 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.553592 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11192947-adab-4eb0-9780-a81e7f8e6356-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.553895 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.553599 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:12.553895 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:12.553608 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t7hw\" (UniqueName: \"kubernetes.io/projected/11192947-adab-4eb0-9780-a81e7f8e6356-kube-api-access-5t7hw\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:13.078261 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.078224 2547 generic.go:356] "Generic (PLEG): container finished" podID="11192947-adab-4eb0-9780-a81e7f8e6356" containerID="ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a" exitCode=0 Jul 23 16:31:13.078419 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.078294 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" Jul 23 16:31:13.078419 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.078311 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" event={"ID":"11192947-adab-4eb0-9780-a81e7f8e6356","Type":"ContainerDied","Data":"ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a"} Jul 23 16:31:13.078419 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.078394 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74" event={"ID":"11192947-adab-4eb0-9780-a81e7f8e6356","Type":"ContainerDied","Data":"ff8a31c6043db9a3b227428f851a71da96cc16268f03c73efcfa4021efcc79f9"} Jul 23 16:31:13.078419 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.078418 2547 scope.go:117] "RemoveContainer" containerID="ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a" Jul 23 16:31:13.087467 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.087453 2547 scope.go:117] "RemoveContainer" containerID="2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12" Jul 23 16:31:13.095239 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.095219 2547 scope.go:117] "RemoveContainer" containerID="ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a" Jul 23 16:31:13.095495 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:31:13.095473 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a\": container with ID starting with ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a not found: ID does not exist" containerID="ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a" Jul 23 16:31:13.095559 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.095503 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a"} err="failed to get container status \"ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a\": rpc error: code = NotFound desc = could not find container \"ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a\": container with ID starting with ebe12bf4c64701372f8d9cc5f49eba17a6161c0569eddddf5ad2e8235e33e27a not found: ID does not exist" Jul 23 16:31:13.095559 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.095521 2547 scope.go:117] "RemoveContainer" containerID="2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12" Jul 23 16:31:13.095788 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:31:13.095765 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12\": container with ID starting with 2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12 not found: ID does not exist" containerID="2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12" Jul 23 16:31:13.095895 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.095795 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12"} err="failed to get container status \"2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12\": rpc error: code = NotFound desc = could not find container \"2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12\": container with ID starting with 2717fa1a17e6bd457db848a5b633e9f0769f854a2aa060c3dd2d41a02de4ee12 not found: ID does not exist" Jul 23 16:31:13.103267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.103246 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74"] Jul 23 16:31:13.106143 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:13.106124 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-dbc842-openshift-default-8b6ff54c7-5zc74"] Jul 23 16:31:14.241304 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:14.241272 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" path="/var/lib/kubelet/pods/11192947-adab-4eb0-9780-a81e7f8e6356/volumes" Jul 23 16:31:34.087883 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.087845 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r"] Jul 23 16:31:34.088500 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.088103 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" containerID="cri-o://e1fe1d9043dc50b00345ebe50b5ca38b8fa07e698ef77882e30b41b957604904" gracePeriod=30 Jul 23 16:31:34.438170 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.438072 2547 patch_prober.go:28] interesting pod/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 16:31:34.438170 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.438161 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 16:31:34.997572 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.997540 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf"] Jul 23 16:31:34.998077 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.998051 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" Jul 23 16:31:34.998077 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.998076 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" Jul 23 16:31:34.998277 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.998105 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" Jul 23 16:31:34.998277 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.998115 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" Jul 23 16:31:34.998277 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:34.998198 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" Jul 23 16:31:35.001278 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.001258 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.003535 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.003514 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-9d26cd-openshift-default-dockercfg-v5544\"" Jul 23 16:31:35.013897 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.013865 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf"] Jul 23 16:31:35.170663 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.170625 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-envoy\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.170688 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-istio-token\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.170816 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdbpb\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-kube-api-access-hdbpb\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.170860 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-certs\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.170932 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5398f75-47d5-4ada-95e9-80783902192d-istiod-ca-cert\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.170956 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-credential-socket\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.170989 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5398f75-47d5-4ada-95e9-80783902192d-istio-podinfo\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.171064 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-socket\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.171117 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.171107 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-data\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.271903 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.271809 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5398f75-47d5-4ada-95e9-80783902192d-istiod-ca-cert\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.271903 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.271846 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-credential-socket\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.271903 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.271866 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5398f75-47d5-4ada-95e9-80783902192d-istio-podinfo\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272210 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272008 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-socket\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272210 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272076 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-data\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272210 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272098 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-envoy\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272210 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272137 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-istio-token\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272210 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272202 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdbpb\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-kube-api-access-hdbpb\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272472 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272239 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-certs\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272472 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272265 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-credential-socket\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272652 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272486 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-data\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272652 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272544 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-certs\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272652 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272596 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-socket\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.272831 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.272740 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5398f75-47d5-4ada-95e9-80783902192d-istiod-ca-cert\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.274210 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.274187 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-envoy\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.274480 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.274465 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5398f75-47d5-4ada-95e9-80783902192d-istio-podinfo\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.280320 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.280290 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-istio-token\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.280552 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.280534 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdbpb\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-kube-api-access-hdbpb\") pod \"e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.312606 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.312577 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:35.454353 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.454322 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf"] Jul 23 16:31:35.456597 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:31:35.456572 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5398f75_47d5_4ada_95e9_80783902192d.slice/crio-ad52394821ad09cb1ccfab106e1c8126cb415065f1fb6d0636f013c14620f59a WatchSource:0}: Error finding container ad52394821ad09cb1ccfab106e1c8126cb415065f1fb6d0636f013c14620f59a: Status 404 returned error can't find the container with id ad52394821ad09cb1ccfab106e1c8126cb415065f1fb6d0636f013c14620f59a Jul 23 16:31:35.458791 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.458758 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:35.458892 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.458827 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:35.458892 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:35.458854 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:36.194887 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:36.194853 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" event={"ID":"e5398f75-47d5-4ada-95e9-80783902192d","Type":"ContainerStarted","Data":"17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456"} Jul 23 16:31:36.195268 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:36.194902 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" event={"ID":"e5398f75-47d5-4ada-95e9-80783902192d","Type":"ContainerStarted","Data":"ad52394821ad09cb1ccfab106e1c8126cb415065f1fb6d0636f013c14620f59a"} Jul 23 16:31:36.215365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:36.215311 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podStartSLOduration=2.215294252 podStartE2EDuration="2.215294252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:31:36.214240765 +0000 UTC m=+2482.490982294" watchObservedRunningTime="2026-07-23 16:31:36.215294252 +0000 UTC m=+2482.492035654" Jul 23 16:31:36.312952 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:36.312921 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:36.317804 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:36.317779 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:37.199739 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:37.199713 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:37.200843 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:37.200824 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:39.212261 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.212228 2547 generic.go:356] "Generic (PLEG): container finished" podID="0170e9f0-4e12-4769-b016-a65450e971a7" containerID="e1fe1d9043dc50b00345ebe50b5ca38b8fa07e698ef77882e30b41b957604904" exitCode=0 Jul 23 16:31:39.212674 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.212306 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" event={"ID":"0170e9f0-4e12-4769-b016-a65450e971a7","Type":"ContainerDied","Data":"e1fe1d9043dc50b00345ebe50b5ca38b8fa07e698ef77882e30b41b957604904"} Jul 23 16:31:39.212674 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.212387 2547 scope.go:117] "RemoveContainer" containerID="6029de3ccabc0746526b59f8903d580cc9668a2134db12fd210a315fa5777172" Jul 23 16:31:39.352654 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.352585 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:31:39.516169 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516132 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-data\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516360 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516195 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-envoy\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516360 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516252 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-credential-socket\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516360 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516352 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0170e9f0-4e12-4769-b016-a65450e971a7-istiod-ca-cert\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516515 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516396 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h59c8\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-kube-api-access-h59c8\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516515 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516422 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0170e9f0-4e12-4769-b016-a65450e971a7-istio-podinfo\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516515 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516462 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-certs\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516515 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516490 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-socket\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516781 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516533 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-istio-token\") pod \"0170e9f0-4e12-4769-b016-a65450e971a7\" (UID: \"0170e9f0-4e12-4769-b016-a65450e971a7\") " Jul 23 16:31:39.516781 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516538 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:39.516781 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516774 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:39.516945 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516781 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0170e9f0-4e12-4769-b016-a65450e971a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:31:39.517002 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516945 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:39.517002 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.516984 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0170e9f0-4e12-4769-b016-a65450e971a7-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.517113 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.517008 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.517113 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.517022 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.517113 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.517084 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:39.518542 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.518514 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-kube-api-access-h59c8" (OuterVolumeSpecName: "kube-api-access-h59c8") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "kube-api-access-h59c8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:31:39.518869 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.518842 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:31:39.518958 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.518864 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:31:39.518958 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.518906 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0170e9f0-4e12-4769-b016-a65450e971a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0170e9f0-4e12-4769-b016-a65450e971a7" (UID: "0170e9f0-4e12-4769-b016-a65450e971a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:31:39.617882 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.617795 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.617882 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.617825 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.617882 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.617834 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h59c8\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-kube-api-access-h59c8\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.617882 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.617846 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0170e9f0-4e12-4769-b016-a65450e971a7-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.617882 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.617855 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0170e9f0-4e12-4769-b016-a65450e971a7-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:39.617882 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:39.617863 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0170e9f0-4e12-4769-b016-a65450e971a7-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:31:40.219215 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:40.219191 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" Jul 23 16:31:40.219707 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:40.219237 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r" event={"ID":"0170e9f0-4e12-4769-b016-a65450e971a7","Type":"ContainerDied","Data":"cca44beb016c82bfca3465f863467344c5384f2d4ebec99fde8db06a7b718659"} Jul 23 16:31:40.219707 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:40.219295 2547 scope.go:117] "RemoveContainer" containerID="e1fe1d9043dc50b00345ebe50b5ca38b8fa07e698ef77882e30b41b957604904" Jul 23 16:31:40.246837 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:40.246809 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r"] Jul 23 16:31:40.250662 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:40.250635 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6fc9aa-openshift-default-85c8c8cf8f-s996r"] Jul 23 16:31:42.240781 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:42.240747 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" path="/var/lib/kubelet/pods/0170e9f0-4e12-4769-b016-a65450e971a7/volumes" Jul 23 16:31:43.365008 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.364974 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs"] Jul 23 16:31:43.365389 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.365376 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" Jul 23 16:31:43.365389 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.365390 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" Jul 23 16:31:43.365464 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.365405 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" Jul 23 16:31:43.365464 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.365411 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" Jul 23 16:31:43.365529 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.365467 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="11192947-adab-4eb0-9780-a81e7f8e6356" containerName="istio-proxy" Jul 23 16:31:43.365529 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.365476 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" Jul 23 16:31:43.370004 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.369984 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.373046 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.373027 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ebb72a-openshift-default-dockercfg-jg89w\"" Jul 23 16:31:43.390340 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.390313 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs"] Jul 23 16:31:43.454274 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454224 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-podinfo\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454471 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454290 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istiod-ca-cert\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454471 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454369 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-certs\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454471 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454425 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-data\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454509 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-credential-socket\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454539 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-socket\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454564 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzcqc\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-kube-api-access-gzcqc\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454680 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454607 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-token\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.454863 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.454709 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-envoy\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.555708 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.555657 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-credential-socket\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.555708 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.555710 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-socket\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.555994 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.555734 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzcqc\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-kube-api-access-gzcqc\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.555994 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.555767 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-token\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.555994 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.555809 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-envoy\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.555994 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.555980 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-podinfo\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556212 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556051 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istiod-ca-cert\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556212 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556091 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-certs\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556212 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556127 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-credential-socket\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556212 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556139 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-data\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556212 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556174 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-socket\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556527 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556400 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-data\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556527 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556469 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-certs\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.556812 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.556777 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istiod-ca-cert\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.558336 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.558311 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-envoy\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.558557 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.558538 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-podinfo\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.564303 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.564281 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-token\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.564954 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.564938 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzcqc\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-kube-api-access-gzcqc\") pod \"e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.680653 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.680528 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:43.814622 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.814598 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs"] Jul 23 16:31:43.816212 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:31:43.816181 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0a5ad5_2062_4ad1_ad7a_d53e238e2acd.slice/crio-dd1b68e877d699eea2c1f0a1df40ccb20080e0263b19d5d6f310cbf65b7a23a5 WatchSource:0}: Error finding container dd1b68e877d699eea2c1f0a1df40ccb20080e0263b19d5d6f310cbf65b7a23a5: Status 404 returned error can't find the container with id dd1b68e877d699eea2c1f0a1df40ccb20080e0263b19d5d6f310cbf65b7a23a5 Jul 23 16:31:43.818524 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.818478 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:43.818602 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.818581 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:43.818651 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:43.818631 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:44.242077 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:44.242045 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" event={"ID":"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd","Type":"ContainerStarted","Data":"4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04"} Jul 23 16:31:44.242279 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:44.242091 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" event={"ID":"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd","Type":"ContainerStarted","Data":"dd1b68e877d699eea2c1f0a1df40ccb20080e0263b19d5d6f310cbf65b7a23a5"} Jul 23 16:31:44.262339 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:44.262296 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podStartSLOduration=1.262282218 podStartE2EDuration="1.262282218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 16:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:31:44.261398464 +0000 UTC m=+2490.538139865" watchObservedRunningTime="2026-07-23 16:31:44.262282218 +0000 UTC m=+2490.539023619" Jul 23 16:31:44.681650 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:44.681620 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:44.686880 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:44.686858 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:45.247231 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:45.247198 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:45.248042 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:45.248023 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:31:53.241848 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:53.241807 2547 patch_prober.go:28] interesting pod/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 16:31:53.242237 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:53.241933 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 16:31:55.300501 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:55.300463 2547 generic.go:356] "Generic (PLEG): container finished" podID="e5398f75-47d5-4ada-95e9-80783902192d" containerID="17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456" exitCode=0 Jul 23 16:31:55.301003 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:55.300535 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" event={"ID":"e5398f75-47d5-4ada-95e9-80783902192d","Type":"ContainerDied","Data":"17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456"} Jul 23 16:31:55.301003 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:55.300936 2547 scope.go:117] "RemoveContainer" containerID="17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456" Jul 23 16:31:55.301915 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:55.301884 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:55.302035 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:55.301948 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:55.302035 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:55.301980 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:31:56.307620 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:56.307589 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" event={"ID":"e5398f75-47d5-4ada-95e9-80783902192d","Type":"ContainerStarted","Data":"772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2"} Jul 23 16:31:56.312972 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:56.312951 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:31:56.313171 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:56.313149 2547 patch_prober.go:28] interesting pod/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 16:31:56.313246 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:56.313202 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 16:31:57.313156 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:57.313114 2547 patch_prober.go:28] interesting pod/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 16:31:57.313528 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:57.313181 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 16:31:58.312978 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:58.312948 2547 patch_prober.go:28] interesting pod/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 16:31:58.313195 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:58.312996 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 16:31:59.313109 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:59.313075 2547 patch_prober.go:28] interesting pod/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 16:31:59.313296 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:31:59.313136 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 16:32:00.313360 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:00.313322 2547 patch_prober.go:28] interesting pod/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 16:32:00.313847 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:00.313381 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 16:32:01.177258 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.177020 2547 patch_prober.go:28] interesting pod/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:59826->10.132.0.44:15021: read: connection reset by peer" start-of-body= Jul 23 16:32:01.177258 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.177086 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:59826->10.132.0.44:15021: read: connection reset by peer" Jul 23 16:32:01.316560 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.316533 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:32:01.316947 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.316841 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:32:01.317498 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.317482 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:32:01.332902 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.332872 2547 generic.go:356] "Generic (PLEG): container finished" podID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerID="4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04" exitCode=0 Jul 23 16:32:01.333018 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.332940 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" event={"ID":"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd","Type":"ContainerDied","Data":"4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04"} Jul 23 16:32:01.333438 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.333421 2547 scope.go:117] "RemoveContainer" containerID="4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04" Jul 23 16:32:01.334408 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.334381 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:32:01.334491 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.334459 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:32:01.334553 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:01.334510 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 16:32:02.348319 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:02.348274 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" event={"ID":"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd","Type":"ContainerStarted","Data":"3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4"} Jul 23 16:32:02.681716 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:02.681619 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:32:02.681874 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:02.681813 2547 patch_prober.go:28] interesting pod/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 16:32:02.681874 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:02.681853 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 16:32:03.681800 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:03.681767 2547 patch_prober.go:28] interesting pod/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 16:32:03.682211 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:03.681818 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 16:32:04.681796 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:04.681758 2547 patch_prober.go:28] interesting pod/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 16:32:04.682038 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:04.681829 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 16:32:05.680929 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:05.680893 2547 patch_prober.go:28] interesting pod/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 16:32:05.681196 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:05.680950 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 16:32:06.684917 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:06.684882 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:32:06.685314 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:06.685220 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:32:06.685827 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:32:06.685811 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:35:14.379256 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:35:14.379178 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:35:14.381582 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:35:14.381557 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:35:14.388026 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:35:14.388007 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:35:14.390075 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:35:14.390059 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:38:09.648280 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:09.648197 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf"] Jul 23 16:38:09.648769 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:09.648431 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" containerID="cri-o://772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2" gracePeriod=30 Jul 23 16:38:14.907218 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:14.907194 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:38:15.080102 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080075 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-envoy\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080299 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080147 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-socket\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080299 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080186 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5398f75-47d5-4ada-95e9-80783902192d-istio-podinfo\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080299 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080223 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdbpb\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-kube-api-access-hdbpb\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080299 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080260 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-data\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080526 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080315 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-credential-socket\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080526 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080344 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5398f75-47d5-4ada-95e9-80783902192d-istiod-ca-cert\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080526 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080403 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-certs\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080526 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080435 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-istio-token\") pod \"e5398f75-47d5-4ada-95e9-80783902192d\" (UID: \"e5398f75-47d5-4ada-95e9-80783902192d\") " Jul 23 16:38:15.080526 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080449 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:15.080889 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080740 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:15.080889 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080786 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5398f75-47d5-4ada-95e9-80783902192d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:38:15.081060 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080887 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:15.081060 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080898 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.081060 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.080917 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.081403 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.081379 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:15.082467 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.082439 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5398f75-47d5-4ada-95e9-80783902192d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:38:15.082598 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.082578 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-kube-api-access-hdbpb" (OuterVolumeSpecName: "kube-api-access-hdbpb") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "kube-api-access-hdbpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:38:15.082761 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.082743 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:15.082854 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.082841 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5398f75-47d5-4ada-95e9-80783902192d" (UID: "e5398f75-47d5-4ada-95e9-80783902192d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:38:15.173994 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.173957 2547 generic.go:356] "Generic (PLEG): container finished" podID="e5398f75-47d5-4ada-95e9-80783902192d" containerID="772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2" exitCode=0 Jul 23 16:38:15.174156 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.174005 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" Jul 23 16:38:15.174156 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.174038 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" event={"ID":"e5398f75-47d5-4ada-95e9-80783902192d","Type":"ContainerDied","Data":"772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2"} Jul 23 16:38:15.174156 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.174090 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf" event={"ID":"e5398f75-47d5-4ada-95e9-80783902192d","Type":"ContainerDied","Data":"ad52394821ad09cb1ccfab106e1c8126cb415065f1fb6d0636f013c14620f59a"} Jul 23 16:38:15.174156 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.174113 2547 scope.go:117] "RemoveContainer" containerID="772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2" Jul 23 16:38:15.181853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.181774 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.181853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.181795 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5398f75-47d5-4ada-95e9-80783902192d-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.181853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.181808 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdbpb\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-kube-api-access-hdbpb\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.181853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.181823 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.181853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.181835 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5398f75-47d5-4ada-95e9-80783902192d-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.181853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.181848 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5398f75-47d5-4ada-95e9-80783902192d-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.182195 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.181860 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5398f75-47d5-4ada-95e9-80783902192d-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:15.183428 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.183408 2547 scope.go:117] "RemoveContainer" containerID="17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456" Jul 23 16:38:15.191323 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.191306 2547 scope.go:117] "RemoveContainer" containerID="772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2" Jul 23 16:38:15.191613 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:38:15.191594 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2\": container with ID starting with 772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2 not found: ID does not exist" containerID="772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2" Jul 23 16:38:15.191662 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.191623 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2"} err="failed to get container status \"772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2\": rpc error: code = NotFound desc = could not find container \"772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2\": container with ID starting with 772ff03770f7b3bd5faceaadd9f350ba5db3485734f03c32262677e47d8ab3d2 not found: ID does not exist" Jul 23 16:38:15.191662 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.191642 2547 scope.go:117] "RemoveContainer" containerID="17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456" Jul 23 16:38:15.191882 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:38:15.191865 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456\": container with ID starting with 17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456 not found: ID does not exist" containerID="17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456" Jul 23 16:38:15.191932 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.191888 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456"} err="failed to get container status \"17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456\": rpc error: code = NotFound desc = could not find container \"17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456\": container with ID starting with 17b0133ec86138dbc5851a2343873a597252c865316ff3bdfe26686d4446e456 not found: ID does not exist" Jul 23 16:38:15.196293 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.196269 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf"] Jul 23 16:38:15.200765 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:15.200739 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-9d26cd-openshift-default-5d9ffb6c7f-zfmtf"] Jul 23 16:38:16.242099 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:16.242057 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5398f75-47d5-4ada-95e9-80783902192d" path="/var/lib/kubelet/pods/e5398f75-47d5-4ada-95e9-80783902192d/volumes" Jul 23 16:38:31.967129 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:31.967096 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs"] Jul 23 16:38:31.967713 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:31.967354 2547 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" containerID="cri-o://3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4" gracePeriod=30 Jul 23 16:38:36.687496 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:36.687458 2547 patch_prober.go:28] interesting pod/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 16:38:36.687878 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:36.687528 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 16:38:37.222555 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.222534 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:38:37.281146 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.281085 2547 generic.go:356] "Generic (PLEG): container finished" podID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerID="3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4" exitCode=0 Jul 23 16:38:37.281146 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.281123 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" Jul 23 16:38:37.281313 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.281145 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" event={"ID":"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd","Type":"ContainerDied","Data":"3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4"} Jul 23 16:38:37.281313 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.281184 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs" event={"ID":"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd","Type":"ContainerDied","Data":"dd1b68e877d699eea2c1f0a1df40ccb20080e0263b19d5d6f310cbf65b7a23a5"} Jul 23 16:38:37.281313 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.281206 2547 scope.go:117] "RemoveContainer" containerID="3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4" Jul 23 16:38:37.289514 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.289498 2547 scope.go:117] "RemoveContainer" containerID="4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04" Jul 23 16:38:37.292574 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292556 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-podinfo\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.292648 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292594 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istiod-ca-cert\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.292648 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292614 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-credential-socket\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.292758 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292654 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-certs\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.292758 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292720 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-socket\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.292863 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292755 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-token\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.292863 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292790 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzcqc\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-kube-api-access-gzcqc\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.292863 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292852 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-data\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.293021 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292891 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-envoy\") pod \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\" (UID: \"cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd\") " Jul 23 16:38:37.293021 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292958 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 16:38:37.293126 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.293084 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:37.293126 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.292958 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:37.293254 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.293226 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:37.293319 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.293290 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.293319 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.293304 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istiod-ca-cert\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.293423 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.293319 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-credential-socket\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.293423 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.293331 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-workload-certs\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.293793 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.293769 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:37.295161 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.295129 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 16:38:37.295279 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.295200 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 16:38:37.295351 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.295335 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:38:37.295501 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.295479 2547 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-kube-api-access-gzcqc" (OuterVolumeSpecName: "kube-api-access-gzcqc") pod "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" (UID: "cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd"). InnerVolumeSpecName "kube-api-access-gzcqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 16:38:37.298113 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.298097 2547 scope.go:117] "RemoveContainer" containerID="3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4" Jul 23 16:38:37.298394 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:38:37.298376 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4\": container with ID starting with 3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4 not found: ID does not exist" containerID="3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4" Jul 23 16:38:37.298450 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.298403 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4"} err="failed to get container status \"3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4\": rpc error: code = NotFound desc = could not find container \"3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4\": container with ID starting with 3a3c5325bdc7bbca33cdd09938d98c9fa9af09a22ecd3edac8f2720776ffc4f4 not found: ID does not exist" Jul 23 16:38:37.298450 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.298421 2547 scope.go:117] "RemoveContainer" containerID="4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04" Jul 23 16:38:37.298754 ip-10-0-139-161 kubenswrapper[2547]: E0723 16:38:37.298672 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04\": container with ID starting with 4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04 not found: ID does not exist" containerID="4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04" Jul 23 16:38:37.298824 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.298751 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04"} err="failed to get container status \"4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04\": rpc error: code = NotFound desc = could not find container \"4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04\": container with ID starting with 4a7bc1eb121ad33fdb28b5986ea04b2508a1e7fa175202ee376b0f0a2ccedd04 not found: ID does not exist" Jul 23 16:38:37.394232 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.394208 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-envoy\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.394232 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.394231 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-podinfo\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.394385 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.394239 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-token\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.394385 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.394249 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzcqc\" (UniqueName: \"kubernetes.io/projected/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-kube-api-access-gzcqc\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.394385 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.394257 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd-istio-data\") on node \"ip-10-0-139-161.ec2.internal\" DevicePath \"\"" Jul 23 16:38:37.602823 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.602793 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs"] Jul 23 16:38:37.606875 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:37.606843 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ebb72a-openshift-default-555dcdb75-txdbs"] Jul 23 16:38:38.240438 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:38.240395 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" path="/var/lib/kubelet/pods/cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd/volumes" Jul 23 16:38:38.796936 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:38.796908 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zjs8r_a08ad834-0797-497a-8945-776dba2807fc/server/0.log" Jul 23 16:38:39.312578 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:39.312550 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-575b84b7c9-wnfd8_9f9bbe05-4189-4121-8e51-89605c902143/copy-manifests/0.log" Jul 23 16:38:39.319217 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:39.319198 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-575b84b7c9-wnfd8_9f9bbe05-4189-4121-8e51-89605c902143/manager/0.log" Jul 23 16:38:42.671288 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:42.671258 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v8c8g_5caef1be-fb54-4ec5-ac5d-7709b1efd240/istio-proxy/0.log" Jul 23 16:38:42.792333 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:42.792309 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gzpsk_2edd8aa4-d17d-4f43-9433-ba0013329544/discovery/0.log" Jul 23 16:38:43.153228 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:43.153203 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz5d4_cd032bd0-d807-489d-8008-47c5bac9f4c2/istio-proxy/1.log" Jul 23 16:38:43.433435 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:43.433356 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hpkln_36aa9b41-18eb-409c-a638-1e611b0647a3/payload-processing/0.log" Jul 23 16:38:43.553047 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:43.553022 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-749854d9c6-jljn4_baff5aa4-e1cc-4e0c-baba-6f2711cdaacf/router/0.log" Jul 23 16:38:43.914365 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:43.914337 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-smdv7_7edf82da-be43-4af5-909a-e523d4351ab3/main/0.log" Jul 23 16:38:44.290467 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:44.290395 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frv8kp_3010b92a-1c4e-44c8-bab1-7626e2a3e0d1/main/0.log" Jul 23 16:38:53.268370 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:53.268342 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gphkh_b38c39bd-bd8b-40e9-95ad-c1ef17a463a0/global-pull-secret-syncer/0.log" Jul 23 16:38:53.449617 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:53.449587 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-78ctg_48d96dfd-fc36-408b-bc0c-02b407eaa2d1/konnectivity-agent/0.log" Jul 23 16:38:53.565424 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:38:53.565392 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-161.ec2.internal_9d6081f961b13beba9ffc30484d6a818/haproxy/0.log" Jul 23 16:39:01.742414 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:01.742371 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-ctbdr_6a3d81a0-7174-498a-be1d-c558568110ee/cluster-monitoring-operator/0.log" Jul 23 16:39:01.799890 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:01.799835 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-j5bv2_e79829f6-6fb4-429b-a94f-0c519dae8c76/kube-state-metrics/0.log" Jul 23 16:39:01.819083 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:01.819029 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-j5bv2_e79829f6-6fb4-429b-a94f-0c519dae8c76/kube-rbac-proxy-main/0.log" Jul 23 16:39:01.841259 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:01.841238 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-j5bv2_e79829f6-6fb4-429b-a94f-0c519dae8c76/kube-rbac-proxy-self/0.log" Jul 23 16:39:02.002973 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.002953 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sxvbj_8f42f8fa-883a-46be-ba8b-f5cc32722ac5/node-exporter/0.log" Jul 23 16:39:02.026134 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.026084 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sxvbj_8f42f8fa-883a-46be-ba8b-f5cc32722ac5/kube-rbac-proxy/0.log" Jul 23 16:39:02.050673 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.050623 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sxvbj_8f42f8fa-883a-46be-ba8b-f5cc32722ac5/init-textfile/0.log" Jul 23 16:39:02.251531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.251494 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff/prometheus/0.log" Jul 23 16:39:02.275122 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.275095 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff/config-reloader/0.log" Jul 23 16:39:02.297343 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.297328 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff/thanos-sidecar/0.log" Jul 23 16:39:02.322214 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.322161 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff/kube-rbac-proxy-web/0.log" Jul 23 16:39:02.345051 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.345034 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff/kube-rbac-proxy/0.log" Jul 23 16:39:02.368183 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.368167 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff/kube-rbac-proxy-thanos/0.log" Jul 23 16:39:02.394121 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.394101 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_39bd265e-ab48-4e5a-93ef-07f4bf1fe6ff/init-config-reloader/0.log" Jul 23 16:39:02.592757 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.592662 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bbc7bb9dd-l5rw7_8acea5af-dc01-432b-8126-070ab1a30b0b/thanos-query/0.log" Jul 23 16:39:02.613947 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.613919 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bbc7bb9dd-l5rw7_8acea5af-dc01-432b-8126-070ab1a30b0b/kube-rbac-proxy-web/0.log" Jul 23 16:39:02.634914 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.634880 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bbc7bb9dd-l5rw7_8acea5af-dc01-432b-8126-070ab1a30b0b/kube-rbac-proxy/0.log" Jul 23 16:39:02.658084 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.658060 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bbc7bb9dd-l5rw7_8acea5af-dc01-432b-8126-070ab1a30b0b/prom-label-proxy/0.log" Jul 23 16:39:02.679494 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.679464 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bbc7bb9dd-l5rw7_8acea5af-dc01-432b-8126-070ab1a30b0b/kube-rbac-proxy-rules/0.log" Jul 23 16:39:02.700535 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:02.700502 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7bbc7bb9dd-l5rw7_8acea5af-dc01-432b-8126-070ab1a30b0b/kube-rbac-proxy-metrics/0.log" Jul 23 16:39:03.835034 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:03.835013 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-4sj5m_561f3cb0-7a0d-4180-b795-504dee568a8e/networking-console-plugin/0.log" Jul 23 16:39:04.430831 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:04.430801 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/1.log" Jul 23 16:39:04.437374 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:04.437343 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-nvzv6_d2c43bce-150e-49ca-b1e9-ff80e7521b67/console-operator/2.log" Jul 23 16:39:05.520167 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:05.520129 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-hqksf_8b97bba7-ca5f-4c1b-b561-848da1d045e7/volume-data-source-validator/0.log" Jul 23 16:39:06.766761 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.766734 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ds84x_2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6/dns/0.log" Jul 23 16:39:06.813517 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.813492 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ds84x_2e3deb1e-d8a6-4752-a5b6-2fa40cc735a6/kube-rbac-proxy/0.log" Jul 23 16:39:06.906613 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.906585 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rrxrl_e7d9a113-8cc2-42e8-807c-58db20ab3221/dns-node-resolver/0.log" Jul 23 16:39:06.959182 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959149 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58"] Jul 23 16:39:06.959653 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959624 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" Jul 23 16:39:06.959653 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959648 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" Jul 23 16:39:06.959878 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959679 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" Jul 23 16:39:06.959878 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959687 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" Jul 23 16:39:06.959878 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959730 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" Jul 23 16:39:06.959878 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959739 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" Jul 23 16:39:06.959878 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959866 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" Jul 23 16:39:06.960051 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959883 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf0a5ad5-2062-4ad1-ad7a-d53e238e2acd" containerName="istio-proxy" Jul 23 16:39:06.960051 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959892 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="0170e9f0-4e12-4769-b016-a65450e971a7" containerName="istio-proxy" Jul 23 16:39:06.960051 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959902 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" Jul 23 16:39:06.960051 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.959909 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5398f75-47d5-4ada-95e9-80783902192d" containerName="istio-proxy" Jul 23 16:39:06.962896 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.962881 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:06.965238 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.965221 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9l5zs\"/\"kube-root-ca.crt\"" Jul 23 16:39:06.965391 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.965374 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9l5zs\"/\"openshift-service-ca.crt\"" Jul 23 16:39:06.966213 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.966195 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9l5zs\"/\"default-dockercfg-ncpq6\"" Jul 23 16:39:06.976140 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:06.976120 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58"] Jul 23 16:39:07.063963 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.063940 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-podres\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.064133 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.063986 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h82pf\" (UniqueName: \"kubernetes.io/projected/07a72156-c1d6-49e2-99ce-e902aa20746c-kube-api-access-h82pf\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.064133 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.064081 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-proc\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.064255 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.064150 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-lib-modules\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.064255 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.064188 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-sys\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.165669 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165639 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-podres\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.165853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165687 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h82pf\" (UniqueName: \"kubernetes.io/projected/07a72156-c1d6-49e2-99ce-e902aa20746c-kube-api-access-h82pf\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.165853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165751 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-proc\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.165853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165803 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-podres\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.165853 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165810 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-lib-modules\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.166028 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165867 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-sys\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.166028 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165902 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-proc\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.166028 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165910 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-lib-modules\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.166028 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.165949 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07a72156-c1d6-49e2-99ce-e902aa20746c-sys\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.183652 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.183623 2547 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h82pf\" (UniqueName: \"kubernetes.io/projected/07a72156-c1d6-49e2-99ce-e902aa20746c-kube-api-access-h82pf\") pod \"perf-node-gather-daemonset-tdj58\" (UID: \"07a72156-c1d6-49e2-99ce-e902aa20746c\") " pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.273205 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.273188 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:07.403332 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.403307 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58"] Jul 23 16:39:07.405086 ip-10-0-139-161 kubenswrapper[2547]: W0723 16:39:07.405054 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod07a72156_c1d6_49e2_99ce_e902aa20746c.slice/crio-bcbc50eb593734e8e7b55b392e3075a3945c387fb9bd46225b3341065f1a7fe5 WatchSource:0}: Error finding container bcbc50eb593734e8e7b55b392e3075a3945c387fb9bd46225b3341065f1a7fe5: Status 404 returned error can't find the container with id bcbc50eb593734e8e7b55b392e3075a3945c387fb9bd46225b3341065f1a7fe5 Jul 23 16:39:07.425047 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.425023 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" event={"ID":"07a72156-c1d6-49e2-99ce-e902aa20746c","Type":"ContainerStarted","Data":"bcbc50eb593734e8e7b55b392e3075a3945c387fb9bd46225b3341065f1a7fe5"} Jul 23 16:39:07.598990 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:07.598926 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lx88p_029afce9-440f-4b8c-992f-97e2786871bf/node-ca/0.log" Jul 23 16:39:08.431412 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.431376 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" event={"ID":"07a72156-c1d6-49e2-99ce-e902aa20746c","Type":"ContainerStarted","Data":"81493a50af8623a5d2e5bf9439b2b267797021eaba68e0e7470d92aaece8536b"} Jul 23 16:39:08.431812 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.431449 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:08.448147 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.448102 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" podStartSLOduration=2.448086613 podStartE2EDuration="2.448086613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 16:39:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 16:39:08.446627725 +0000 UTC m=+2934.723369126" watchObservedRunningTime="2026-07-23 16:39:08.448086613 +0000 UTC m=+2934.724828014" Jul 23 16:39:08.612510 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.612483 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v8c8g_5caef1be-fb54-4ec5-ac5d-7709b1efd240/istio-proxy/0.log" Jul 23 16:39:08.739839 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.739768 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gzpsk_2edd8aa4-d17d-4f43-9433-ba0013329544/discovery/0.log" Jul 23 16:39:08.818340 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.818308 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz5d4_cd032bd0-d807-489d-8008-47c5bac9f4c2/istio-proxy/0.log" Jul 23 16:39:08.848737 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.848708 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz5d4_cd032bd0-d807-489d-8008-47c5bac9f4c2/istio-proxy/1.log" Jul 23 16:39:08.924678 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.924653 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hpkln_36aa9b41-18eb-409c-a638-1e611b0647a3/payload-processing/0.log" Jul 23 16:39:08.947527 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:08.947504 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-749854d9c6-jljn4_baff5aa4-e1cc-4e0c-baba-6f2711cdaacf/router/0.log" Jul 23 16:39:09.722531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:09.722502 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sbmkn_65e6104e-d595-4c17-9afd-685efbb4deca/serve-healthcheck-canary/0.log" Jul 23 16:39:10.225593 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:10.225566 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-sztxq_a8766364-a437-4a90-9058-e1c2549638dc/insights-operator/0.log" Jul 23 16:39:10.226592 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:10.226572 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-sztxq_a8766364-a437-4a90-9058-e1c2549638dc/insights-operator/1.log" Jul 23 16:39:10.398834 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:10.398811 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-txdfw_6170b78e-67ba-4e41-9ff7-95b1602719da/kube-rbac-proxy/0.log" Jul 23 16:39:10.421738 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:10.421709 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-txdfw_6170b78e-67ba-4e41-9ff7-95b1602719da/exporter/0.log" Jul 23 16:39:10.446775 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:10.446749 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-txdfw_6170b78e-67ba-4e41-9ff7-95b1602719da/extractor/0.log" Jul 23 16:39:13.201276 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:13.201251 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zjs8r_a08ad834-0797-497a-8945-776dba2807fc/server/0.log" Jul 23 16:39:13.554956 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:13.554930 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-575b84b7c9-wnfd8_9f9bbe05-4189-4121-8e51-89605c902143/manager/0.log" Jul 23 16:39:13.575840 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:13.575815 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-575b84b7c9-wnfd8_9f9bbe05-4189-4121-8e51-89605c902143/copy-manifests/0.log" Jul 23 16:39:14.134978 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.134952 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v8c8g_5caef1be-fb54-4ec5-ac5d-7709b1efd240/istio-proxy/0.log" Jul 23 16:39:14.250267 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.250245 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gzpsk_2edd8aa4-d17d-4f43-9433-ba0013329544/discovery/0.log" Jul 23 16:39:14.290035 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.290008 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz5d4_cd032bd0-d807-489d-8008-47c5bac9f4c2/istio-proxy/0.log" Jul 23 16:39:14.315098 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.315075 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz5d4_cd032bd0-d807-489d-8008-47c5bac9f4c2/istio-proxy/1.log" Jul 23 16:39:14.366347 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.366320 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hpkln_36aa9b41-18eb-409c-a638-1e611b0647a3/payload-processing/0.log" Jul 23 16:39:14.386877 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.386812 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-749854d9c6-jljn4_baff5aa4-e1cc-4e0c-baba-6f2711cdaacf/router/0.log" Jul 23 16:39:14.445386 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.445362 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9l5zs/perf-node-gather-daemonset-tdj58" Jul 23 16:39:14.767193 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:14.767113 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-775bb6b8db-dm4hq_131efb8c-9532-4b69-92b0-7369885c28a2/manager/0.log" Jul 23 16:39:15.609546 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:15.609517 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4v8c8g_5caef1be-fb54-4ec5-ac5d-7709b1efd240/istio-proxy/0.log" Jul 23 16:39:15.732235 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:15.732204 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gzpsk_2edd8aa4-d17d-4f43-9433-ba0013329544/discovery/0.log" Jul 23 16:39:15.785446 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:15.785422 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz5d4_cd032bd0-d807-489d-8008-47c5bac9f4c2/istio-proxy/0.log" Jul 23 16:39:15.810260 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:15.810241 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tz5d4_cd032bd0-d807-489d-8008-47c5bac9f4c2/istio-proxy/1.log" Jul 23 16:39:15.865311 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:15.865253 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-hpkln_36aa9b41-18eb-409c-a638-1e611b0647a3/payload-processing/0.log" Jul 23 16:39:15.885829 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:15.885811 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-749854d9c6-jljn4_baff5aa4-e1cc-4e0c-baba-6f2711cdaacf/router/0.log" Jul 23 16:39:20.537801 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:20.537772 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-qsbp4_9c75e171-26c0-455f-9380-f4a9d348fcda/kube-storage-version-migrator-operator/1.log" Jul 23 16:39:20.538925 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:20.538902 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-qsbp4_9c75e171-26c0-455f-9380-f4a9d348fcda/kube-storage-version-migrator-operator/0.log" Jul 23 16:39:21.825564 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:21.825538 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql7xx_f76a8e9b-f800-4ecc-ad18-08d9df0c8096/kube-multus-additional-cni-plugins/0.log" Jul 23 16:39:21.848148 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:21.848124 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql7xx_f76a8e9b-f800-4ecc-ad18-08d9df0c8096/egress-router-binary-copy/0.log" Jul 23 16:39:21.870871 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:21.870846 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql7xx_f76a8e9b-f800-4ecc-ad18-08d9df0c8096/cni-plugins/0.log" Jul 23 16:39:21.891605 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:21.891590 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql7xx_f76a8e9b-f800-4ecc-ad18-08d9df0c8096/bond-cni-plugin/0.log" Jul 23 16:39:21.911789 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:21.911751 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql7xx_f76a8e9b-f800-4ecc-ad18-08d9df0c8096/routeoverride-cni/0.log" Jul 23 16:39:21.933252 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:21.933229 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql7xx_f76a8e9b-f800-4ecc-ad18-08d9df0c8096/whereabouts-cni-bincopy/0.log" Jul 23 16:39:21.956092 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:21.956074 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ql7xx_f76a8e9b-f800-4ecc-ad18-08d9df0c8096/whereabouts-cni/0.log" Jul 23 16:39:22.148482 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:22.148418 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qvxz2_f1e6c795-dcda-450f-b818-06d15f4a1a48/kube-multus/0.log" Jul 23 16:39:22.296461 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:22.296433 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sg68n_d199f478-aba8-452c-9224-ed736a726a29/network-metrics-daemon/0.log" Jul 23 16:39:22.328063 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:22.328038 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sg68n_d199f478-aba8-452c-9224-ed736a726a29/kube-rbac-proxy/0.log" Jul 23 16:39:23.470485 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.470455 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-controller/0.log" Jul 23 16:39:23.489188 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.489165 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/1.log" Jul 23 16:39:23.489306 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.489205 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovn-acl-logging/0.log" Jul 23 16:39:23.506453 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.506430 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/kube-rbac-proxy-node/0.log" Jul 23 16:39:23.529574 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.529557 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/kube-rbac-proxy-ovn-metrics/0.log" Jul 23 16:39:23.548230 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.548211 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/northd/0.log" Jul 23 16:39:23.568531 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.568510 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/nbdb/0.log" Jul 23 16:39:23.588937 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.588917 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/sbdb/0.log" Jul 23 16:39:23.641142 ip-10-0-139-161 kubenswrapper[2547]: I0723 16:39:23.641124 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k4qjk_c102b23d-0f75-4ac7-b4d2-d7ba8f48a734/ovnkube-controller/0.log"