Jul 06 17:38:27.752592 ip-10-0-140-115 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 17:38:27.752607 ip-10-0-140-115 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 17:38:27.752616 ip-10-0-140-115 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 17:38:27.752951 ip-10-0-140-115 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 17:38:37.954091 ip-10-0-140-115 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 17:38:37.954107 ip-10-0-140-115 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 06cbe1a217bc4bb1a3fd15c21f951d1d -- Jul 06 17:40:32.484727 ip-10-0-140-115 systemd[1]: Starting Kubernetes Kubelet... Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.813799 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817051 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817064 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817067 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817071 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817074 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817077 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817080 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:40:32.936021 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817082 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:40:32.883057 ip-10-0-140-115 systemd[1]: Started Kubernetes Kubelet. Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817087 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817090 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817093 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817096 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817099 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817102 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817105 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817107 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817110 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817113 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817115 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817118 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817120 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817125 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817129 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817132 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817135 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817138 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:40:32.937414 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817141 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817144 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817146 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817149 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817152 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817154 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817157 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817160 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817162 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817165 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817167 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817170 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817173 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817175 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817179 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817181 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817184 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817187 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817191 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817194 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:40:32.942095 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817196 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817199 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817202 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817204 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817207 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817210 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817213 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817215 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817218 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817221 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817223 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817226 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817229 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817231 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817234 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817239 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817243 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817246 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817249 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817252 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:40:32.943053 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817255 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817258 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817261 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817264 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817268 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817271 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817274 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817277 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817279 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817282 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817284 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817287 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817289 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817292 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817295 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817297 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817300 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817304 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817308 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:40:32.944153 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817311 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817313 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817316 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817318 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817321 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817323 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817326 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817329 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817331 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817334 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817337 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817340 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817342 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.817345 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818732 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818739 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818742 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818754 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818757 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818760 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:40:32.945082 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818763 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818765 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818769 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818771 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818774 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818776 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818779 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818781 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818784 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818787 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818789 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818792 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818795 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818797 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818801 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818804 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818807 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818809 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818812 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:40:32.946828 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818814 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818817 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818819 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818823 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818828 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818830 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818833 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818835 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818838 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818840 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818842 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818846 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818848 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818851 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818853 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818856 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818860 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818864 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818867 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:40:32.948408 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818870 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818872 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818875 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818878 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818881 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818883 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818886 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818889 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818892 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818894 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818897 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818900 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818903 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818905 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818908 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818910 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818912 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818914 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818917 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:40:32.949374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818920 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818923 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818925 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818928 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818930 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818935 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818938 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818942 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818945 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818948 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818952 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818955 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818957 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818960 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818962 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818965 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818967 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818970 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818972 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818975 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:40:32.952420 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818977 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818980 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818982 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818984 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818987 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818990 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818992 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818995 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.818997 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819000 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819002 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819005 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819007 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819009 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819012 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819015 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819379 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819384 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819387 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819390 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:40:32.954783 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819392 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819396 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819398 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819401 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819404 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819406 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819409 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819412 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819414 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819417 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819419 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819422 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819440 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819443 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819446 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819448 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819452 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819455 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819457 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819460 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:40:32.956912 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819462 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819465 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819467 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819470 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819473 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819476 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819478 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819482 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819484 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819487 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819489 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819492 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819495 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819498 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819501 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819503 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819506 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819508 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819511 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819513 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:40:32.957936 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819515 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819518 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819520 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819523 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819525 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819541 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819543 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819546 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819549 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819551 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819554 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819557 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819560 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819562 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819565 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819567 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819571 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819578 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819581 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819584 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:40:32.959697 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819587 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819590 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819593 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819596 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819600 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819603 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819605 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819608 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819610 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819613 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819616 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819619 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819623 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819626 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819628 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819631 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819635 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819638 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819641 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:40:32.961094 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819645 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819648 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819651 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819654 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819657 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819659 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819662 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819664 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819667 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819669 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819675 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819677 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819680 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819684 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819687 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.819690 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819756 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819762 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819770 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819774 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819779 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 17:40:32.961942 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819782 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819786 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819790 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819794 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819798 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819801 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819804 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819807 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819810 2558 flags.go:64] FLAG: --cgroup-root="" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819813 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819816 2558 flags.go:64] FLAG: --client-ca-file="" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819819 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819822 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819826 2558 flags.go:64] FLAG: --cluster-domain="" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819829 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819832 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819835 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819838 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819842 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819845 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819848 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819852 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819855 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819858 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 17:40:32.962994 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819861 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819864 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819867 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819871 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819874 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819877 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819880 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819883 2558 flags.go:64] FLAG: --enable-server="true" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819886 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819890 2558 flags.go:64] FLAG: --event-burst="100" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819893 2558 flags.go:64] FLAG: --event-qps="50" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819895 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819899 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819902 2558 flags.go:64] FLAG: --eviction-hard="" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819906 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819909 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819911 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819914 2558 flags.go:64] FLAG: --eviction-soft="" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819917 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819920 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819923 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819925 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819928 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819931 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819934 2558 flags.go:64] FLAG: --feature-gates="" Jul 06 17:40:32.964016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819938 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819940 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819943 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819946 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819951 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819954 2558 flags.go:64] FLAG: --help="false" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819956 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-140-115.ec2.internal" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819959 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819963 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819966 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819969 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819972 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819975 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819978 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819981 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819984 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819987 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819990 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819993 2558 flags.go:64] FLAG: --kube-reserved="" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.819997 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820000 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820002 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820005 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820008 2558 flags.go:64] FLAG: --lock-file="" Jul 06 17:40:32.964857 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820011 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820014 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820017 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820022 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820025 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820027 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820030 2558 flags.go:64] FLAG: --logging-format="text" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820033 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820036 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820038 2558 flags.go:64] FLAG: --manifest-url="" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820041 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820045 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820049 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820053 2558 flags.go:64] FLAG: --max-pods="110" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820056 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820058 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820061 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820064 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820067 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820069 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820072 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820079 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820081 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820085 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 17:40:32.965873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820088 2558 flags.go:64] FLAG: --pod-cidr="" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820091 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820096 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820099 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820103 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820106 2558 flags.go:64] FLAG: --port="10250" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820108 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820111 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02904b47572df784e" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820114 2558 flags.go:64] FLAG: --qos-reserved="" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820117 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820120 2558 flags.go:64] FLAG: --register-node="true" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820123 2558 flags.go:64] FLAG: --register-with-taints="" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820126 2558 flags.go:64] FLAG: --registry-burst="10" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820129 2558 flags.go:64] FLAG: --registry-qps="5" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820132 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820134 2558 flags.go:64] FLAG: --reserved-memory="" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820138 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820140 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820143 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820146 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820150 2558 flags.go:64] FLAG: --runonce="false" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820153 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820156 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820159 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820161 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 17:40:32.966696 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820164 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820167 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820170 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820173 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820176 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820179 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820182 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820185 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820188 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820191 2558 flags.go:64] FLAG: --system-cgroups="" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820194 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820200 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820203 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820206 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820210 2558 flags.go:64] FLAG: --tls-min-version="" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820213 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820216 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820218 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820221 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820224 2558 flags.go:64] FLAG: --v="2" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820228 2558 flags.go:64] FLAG: --version="false" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820232 2558 flags.go:64] FLAG: --vmodule="" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820235 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820238 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820350 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:40:32.967542 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820353 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820356 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820361 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820364 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820366 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820369 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820372 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820375 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820377 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820380 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820383 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820386 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820388 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820390 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820393 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820396 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820398 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820401 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820405 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820407 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:40:32.968592 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820410 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820412 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820415 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820417 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820420 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820422 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820441 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820443 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820446 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820449 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820451 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820455 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820459 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820462 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820466 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820469 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820472 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820474 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820477 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:40:32.969374 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820479 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820482 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820484 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820487 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820489 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820503 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820506 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820509 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820511 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820515 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820518 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820523 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820545 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820549 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820551 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820554 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820557 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820560 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820563 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820565 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:40:33.115840 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820568 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820570 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820573 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820576 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820579 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820581 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820584 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820587 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820590 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820593 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820595 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820598 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820601 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820603 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820606 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820608 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820611 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820613 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820616 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820619 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:40:33.130757 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820621 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820624 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820627 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820631 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820634 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820637 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820639 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820642 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820645 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820647 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820650 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820653 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820657 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820660 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820663 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820666 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820669 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820672 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:40:33.131383 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.820675 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.820685 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.829323 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.829345 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.829959 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.829988 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830263 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830273 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830277 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830280 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830284 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830287 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830290 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830293 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830296 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:40:33.132183 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830299 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830301 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830306 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830310 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830313 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830317 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830320 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830323 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830326 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830329 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830332 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830335 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830338 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830340 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830343 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830346 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830348 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830355 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830357 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:40:33.132793 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830360 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830363 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830366 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830369 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830373 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830375 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830378 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830380 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830383 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830385 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830388 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830390 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830393 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830395 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830399 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830402 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830404 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830407 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830409 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830414 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:40:33.133403 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830417 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830419 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830422 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830439 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830443 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830446 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830448 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830451 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830454 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830456 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830460 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830463 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830465 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830469 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830473 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830476 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830479 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830482 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830484 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830487 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:40:33.134014 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830489 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830491 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830494 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830496 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830499 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830502 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830504 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830507 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830509 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830512 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830514 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830520 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830523 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830525 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830528 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830530 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830532 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830535 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830537 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830540 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:40:33.134631 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830543 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830545 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830548 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830552 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830555 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830558 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830561 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830565 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830568 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830571 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830574 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.830579 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830697 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830701 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830704 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 17:40:33.135178 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830707 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830710 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830713 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830716 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830719 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830722 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830725 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830727 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830731 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830733 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830736 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830739 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830741 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830744 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830747 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830749 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830752 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830754 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830756 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 17:40:33.135674 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830759 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830762 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830765 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830767 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830770 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830772 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830775 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830777 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830781 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830784 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830787 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830790 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830793 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830796 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830799 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830802 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830805 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830807 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830809 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 17:40:33.136192 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830812 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830814 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830817 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830820 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830822 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830825 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830827 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830830 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830833 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830835 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830838 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830840 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830843 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830845 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830849 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830852 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830854 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830857 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830859 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830862 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 17:40:33.136789 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830865 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830867 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830870 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830872 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830875 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830877 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830880 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830882 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830884 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830887 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830889 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830892 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830894 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830897 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830900 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830903 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830905 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830910 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830914 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830917 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 17:40:33.137325 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830919 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830922 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830925 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830928 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830930 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830933 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830936 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830939 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830942 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830944 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830948 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830951 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830954 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830956 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830959 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830962 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830964 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 17:40:33.137943 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:32.830966 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.830971 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.831102 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.834488 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.835637 2558 server.go:1019] "Starting client certificate rotation" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.835735 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.835764 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.837758 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.839021 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.848248 2558 log.go:25] "Validated CRI v1 runtime API" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.850307 2558 log.go:25] "Validated CRI v1 image API" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.853415 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 17:40:33.138399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.857464 2558 fs.go:135] Filesystem UUIDs: map[6f0c1bcb-ad17-426c-8389-413f04681053:/dev/nvme0n1p3 75f497b1-d403-48db-acdf-6cd889da57ec:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 06 17:40:33.138804 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.857483 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 17:40:33.138874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.862788 2558 manager.go:216] Machine: {Timestamp:2026-07-06 17:40:32.861053688 +0000 UTC m=+0.266586605 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104997 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec219fc157fb45f6887ad74cc4c4af41 SystemUUID:ec219fc1-57fb-45f6-887a-d74cc4c4af41 BootID:06cbe1a2-17bc-4bb1-a3fd-15c21f951d1d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:48:00:c5:30:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:48:00:c5:30:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:4a:bd:dc:59:65 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 17:40:33.138874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.862896 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 17:40:33.138874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.862973 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 17:40:33.138874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.864509 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 17:40:33.138874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.865339 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 17:40:33.138874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.865364 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-115.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 17:40:33.138874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.866001 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.866010 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.866018 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.866535 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.867667 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.870203 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.870302 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.870328 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.870337 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.870350 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.870358 2558 kubelet.go:398] "Adding apiserver pod source" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.870367 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.871680 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.871719 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.875122 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.877180 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.877207 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878160 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878173 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878179 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878188 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878201 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878209 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878219 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878225 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878232 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878238 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878247 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878252 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878834 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.878841 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.881190 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-115.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.881223 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.881248 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.882264 2558 server.go:1268] "Started kubelet" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.882342 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.882383 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.882407 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.883861 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 17:40:33.139157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.884162 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.887912 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.889401 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.889409 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.889479 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.889968 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.889991 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.889996 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.890087 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.890096 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.890569 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.890950 2558 factory.go:55] Registering systemd factory Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.890962 2558 factory.go:223] Registration of the systemd container factory successfully Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.890288 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-115.ec2.internal.18bfc42330b4276b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-115.ec2.internal,UID:ip-10-0-140-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-115.ec2.internal,},FirstTimestamp:2026-07-06 17:40:32.882018155 +0000 UTC m=+0.287551070,LastTimestamp:2026-07-06 17:40:32.882018155 +0000 UTC m=+0.287551070,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-115.ec2.internal,}" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.891145 2558 factory.go:153] Registering CRI-O factory Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.891158 2558 factory.go:223] Registration of the crio container factory successfully Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.891201 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.891226 2558 factory.go:103] Registering Raw factory Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.891241 2558 manager.go:1196] Started watching for new ooms in manager Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.891644 2558 manager.go:349] Starting recovery of all containers Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.900731 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.904881 2558 manager.go:354] Recovery completed Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.906439 2558 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.909303 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.911166 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.911196 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.911210 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.911856 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.911867 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.911879 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.911898 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.912517 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:40:33.140197 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.912543 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.912570 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.913233 2558 policy_none.go:49] "None policy: Start" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.913244 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.913254 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.914177 2558 policy_none.go:47] "Start" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.921120 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-115.ec2.internal.18bfc423327125d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-115.ec2.internal,UID:ip-10-0-140-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-115.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-115.ec2.internal,},FirstTimestamp:2026-07-06 17:40:32.911181266 +0000 UTC m=+0.316714182,LastTimestamp:2026-07-06 17:40:32.911181266 +0000 UTC m=+0.316714182,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-115.ec2.internal,}" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.922860 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-115.ec2.internal.18bfc42332717fb4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-115.ec2.internal,UID:ip-10-0-140-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-115.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-115.ec2.internal,},FirstTimestamp:2026-07-06 17:40:32.911204276 +0000 UTC m=+0.316737307,LastTimestamp:2026-07-06 17:40:32.911204276 +0000 UTC m=+0.316737307,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-115.ec2.internal,}" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.931322 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-115.ec2.internal.18bfc4233271a8f4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-115.ec2.internal,UID:ip-10-0-140-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-115.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-115.ec2.internal,},FirstTimestamp:2026-07-06 17:40:32.911214836 +0000 UTC m=+0.316747755,LastTimestamp:2026-07-06 17:40:32.911214836 +0000 UTC m=+0.316747755,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-115.ec2.internal,}" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.934780 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-115.ec2.internal.18bfc423327125d2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-115.ec2.internal.18bfc423327125d2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-115.ec2.internal,UID:ip-10-0-140-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-115.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-115.ec2.internal,},FirstTimestamp:2026-07-06 17:40:32.911181266 +0000 UTC m=+0.316714182,LastTimestamp:2026-07-06 17:40:32.912528498 +0000 UTC m=+0.318061417,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-115.ec2.internal,}" Jul 06 17:40:33.141141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.936356 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8t7h5" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.939637 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-115.ec2.internal.18bfc42332717fb4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-115.ec2.internal.18bfc42332717fb4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-115.ec2.internal,UID:ip-10-0-140-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-115.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-115.ec2.internal,},FirstTimestamp:2026-07-06 17:40:32.911204276 +0000 UTC m=+0.316737307,LastTimestamp:2026-07-06 17:40:32.912548081 +0000 UTC m=+0.318080997,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-115.ec2.internal,}" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.943676 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8t7h5" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.945640 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-115.ec2.internal.18bfc4233271a8f4\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-115.ec2.internal.18bfc4233271a8f4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-115.ec2.internal,UID:ip-10-0-140-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-115.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-115.ec2.internal,},FirstTimestamp:2026-07-06 17:40:32.911214836 +0000 UTC m=+0.316747755,LastTimestamp:2026-07-06 17:40:32.912574224 +0000 UTC m=+0.318107141,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-115.ec2.internal,}" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.967987 2558 manager.go:337] "Starting Device Plugin manager" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.968018 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.968031 2558 server.go:87] "Starting device plugin registration server" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.968296 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.968308 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.968407 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.968503 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:32.968512 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.969071 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:32.969146 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.062956 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.064142 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.064178 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.064203 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.064243 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.066990 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.069021 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.069894 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.069928 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.069941 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.069971 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.141976 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.078905 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.142714 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.078935 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-115.ec2.internal\": node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.142714 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.111023 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.164892 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal"] Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.164994 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.165949 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.165980 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.165998 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.167207 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.167317 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.167343 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.168006 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.168027 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.168040 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.168048 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.168069 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.168078 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.169248 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.169278 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.170456 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientMemory" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.170482 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.170497 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeHasSufficientPID" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.191586 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2fe29e2175c9c75acb15d1d4b57872d2-config\") pod \"kube-apiserver-proxy-ip-10-0-140-115.ec2.internal\" (UID: \"2fe29e2175c9c75acb15d1d4b57872d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.191612 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/548ab702b5d1e4dcb62c99874dd47f27-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal\" (UID: \"548ab702b5d1e4dcb62c99874dd47f27\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.191630 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548ab702b5d1e4dcb62c99874dd47f27-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal\" (UID: \"548ab702b5d1e4dcb62c99874dd47f27\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.201351 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-115.ec2.internal\" not found" node="ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.205883 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-115.ec2.internal\" not found" node="ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.296034 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.211921 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.297638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.297496 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/548ab702b5d1e4dcb62c99874dd47f27-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal\" (UID: \"548ab702b5d1e4dcb62c99874dd47f27\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.297638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.297527 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548ab702b5d1e4dcb62c99874dd47f27-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal\" (UID: \"548ab702b5d1e4dcb62c99874dd47f27\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.297638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.297547 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2fe29e2175c9c75acb15d1d4b57872d2-config\") pod \"kube-apiserver-proxy-ip-10-0-140-115.ec2.internal\" (UID: \"2fe29e2175c9c75acb15d1d4b57872d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.297638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.297591 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2fe29e2175c9c75acb15d1d4b57872d2-config\") pod \"kube-apiserver-proxy-ip-10-0-140-115.ec2.internal\" (UID: \"2fe29e2175c9c75acb15d1d4b57872d2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.297638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.297600 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548ab702b5d1e4dcb62c99874dd47f27-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal\" (UID: \"548ab702b5d1e4dcb62c99874dd47f27\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.297638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.297600 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/548ab702b5d1e4dcb62c99874dd47f27-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal\" (UID: \"548ab702b5d1e4dcb62c99874dd47f27\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.445279 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.312807 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.445279 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.413292 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.606064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.504301 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.606064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.509474 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" Jul 06 17:40:33.606064 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.514309 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.731224 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.615677 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.731224 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.716400 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.876163 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.816888 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-115.ec2.internal\" not found" Jul 06 17:40:33.876163 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.836369 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 17:40:33.876163 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.836567 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 17:40:33.876163 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.855556 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:40:33.876163 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.871382 2558 apiserver.go:52] "Watching apiserver" Jul 06 17:40:33.882221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.880450 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 17:40:33.882221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.880888 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-vwrlp","openshift-image-registry/node-ca-th6lw","openshift-multus/multus-additional-cni-plugins-4bqh7","openshift-multus/network-metrics-daemon-b9gd9","openshift-network-diagnostics/network-check-target-6f44w","openshift-ovn-kubernetes/ovnkube-node-z4cwb","kube-system/konnectivity-agent-s65dx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq","openshift-dns/node-resolver-vx2ng","openshift-multus/multus-bzvd4","openshift-network-operator/iptables-alerter-mfvhl","kube-system/global-pull-secret-syncer-vtk7l"] Jul 06 17:40:33.883409 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.882411 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.883784 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.883859 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.884892 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zf5v8\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.884909 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.884915 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.885067 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.885120 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.885237 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.885523 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.887320 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.887614 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.887622 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 17:40:33.888174 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.888094 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g6k2d\"" Jul 06 17:40:33.889094 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.888809 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.889482 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.890165 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.890390 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.891095 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.891202 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-49dxz\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.891790 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.892527 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.892695 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hbxhk\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.892788 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.894004 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.894067 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.894111 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rvh4q\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.894064 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.894170 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.894207 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.896027 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.898037 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.898346 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.898538 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.898552 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.898758 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.898839 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.898863 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.899020 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k4blr\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.899850 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900025 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900227 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kqd5g\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900317 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 17:40:34.001181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900366 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900639 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-etc-selinux\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900660 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-sys-fs\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900680 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-os-release\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900703 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-cni-bin\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900728 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-multus-certs\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900763 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-etc-kubernetes\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900808 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-run\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900830 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-os-release\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900850 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900871 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900900 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-system-cni-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900921 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-kubelet\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900943 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-systemd\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.003033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900957 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/014ceecb-1938-4660-b2fa-7c17bb94b782-host\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.900975 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/014ceecb-1938-4660-b2fa-7c17bb94b782-serviceca\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901025 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901054 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901077 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-socket-dir-parent\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-daemon-config\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901171 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysconfig\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901192 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-sys\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901218 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-lib-modules\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901252 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcs4v\" (UniqueName: \"kubernetes.io/projected/d1dde903-65f8-4917-97c5-6397a9f48c75-kube-api-access-fcs4v\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901269 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-hostroot\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901302 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-conf-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901328 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-kubernetes\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901357 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysctl-d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.005170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901391 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-var-lib-kubelet\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901412 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-host\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901443 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-cnibin\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901469 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-k8s-cni-cncf-io\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901492 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901495 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhkmj\" (UniqueName: \"kubernetes.io/projected/676d07a9-8585-4002-864f-d9c0d0d1e336-kube-api-access-lhkmj\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901587 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901596 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-tuned\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901621 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebeaae4-97b7-459a-9922-1435ab68b1a9-tmp\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901657 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kds2d\" (UniqueName: \"kubernetes.io/projected/3ebeaae4-97b7-459a-9922-1435ab68b1a9-kube-api-access-kds2d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901672 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-system-cni-dir\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901711 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwmsx\" (UniqueName: \"kubernetes.io/projected/8a8ceb14-f428-4d2d-ac73-5c1a777be296-kube-api-access-gwmsx\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901755 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-cni-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901812 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-netns\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901836 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-546lp\" (UniqueName: \"kubernetes.io/projected/014ceecb-1938-4660-b2fa-7c17bb94b782-kube-api-access-546lp\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.006050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901855 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-socket-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901881 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-device-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901904 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/676d07a9-8585-4002-864f-d9c0d0d1e336-cni-binary-copy\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901919 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-cni-multus\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901942 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901946 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysctl-conf\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.901986 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.902008 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-registration-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.902023 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-cnibin\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.902041 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-modprobe-d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.902074 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.902245 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.902275 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sht4m\"" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.902504 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.904528 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.904547 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.904550 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.904587 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8qb8p\"" Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.904690 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal"] Jul 06 17:40:34.007323 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.904796 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:33.904849 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.914424 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal"] Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.914571 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.926984 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dhktq" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.935208 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dhktq" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.945917 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 17:35:33 +0000 UTC" deadline="2028-03-09 20:46:20.899136764 +0000 UTC" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.945964 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14691h5m46.953175402s" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:33.991673 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002396 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-systemd-units\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002421 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-run-netns\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002456 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64899c8b-c773-4375-a064-80b0de31b920-ovn-node-metrics-cert\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002479 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002496 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-registration-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002519 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-cnibin\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002542 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-var-lib-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002610 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-registration-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.008302 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002637 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-env-overrides\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002674 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6bpk\" (UniqueName: \"kubernetes.io/projected/64899c8b-c773-4375-a064-80b0de31b920-kube-api-access-p6bpk\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002718 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-etc-selinux\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002745 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-cnibin\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002747 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-cni-bin\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002775 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-cni-bin\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002776 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-multus-certs\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002802 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-multus-certs\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002811 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-os-release\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002833 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002839 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-etc-selinux\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002850 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-kubelet\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002882 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-systemd\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-kubelet\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.010404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002902 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-os-release\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-systemd\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002950 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-546lp\" (UniqueName: \"kubernetes.io/projected/014ceecb-1938-4660-b2fa-7c17bb94b782-kube-api-access-546lp\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002949 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-systemd\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.002971 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003012 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-socket-dir-parent\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-daemon-config\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysconfig\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-var-lib-kubelet\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003095 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcs4v\" (UniqueName: \"kubernetes.io/projected/d1dde903-65f8-4917-97c5-6397a9f48c75-kube-api-access-fcs4v\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-hostroot\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-kubernetes\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003159 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysctl-d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003182 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-socket-dir-parent\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.011332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003187 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f5de882-ee4b-4c48-bd5e-87726ab35b72-kubelet-config\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003210 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysconfig\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003225 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-var-lib-kubelet\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003215 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f5de882-ee4b-4c48-bd5e-87726ab35b72-dbus\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003158 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003266 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-slash\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003274 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-kubernetes\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003296 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-hostroot\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003299 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-cnibin\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a25767e7-609c-42a7-8b8b-f793ee69d50f-tmp-dir\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003355 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-ovnkube-script-lib\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003333 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-cnibin\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003380 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4879c2c5-4dae-48c7-85b5-f010d8607f1e-konnectivity-ca\") pod \"konnectivity-agent-s65dx\" (UID: \"4879c2c5-4dae-48c7-85b5-f010d8607f1e\") " pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003410 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-system-cni-dir\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.012053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003441 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwmsx\" (UniqueName: \"kubernetes.io/projected/8a8ceb14-f428-4d2d-ac73-5c1a777be296-kube-api-access-gwmsx\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003466 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d1dde903-65f8-4917-97c5-6397a9f48c75-system-cni-dir\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003471 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysctl-d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003466 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kkj6\" (UniqueName: \"kubernetes.io/projected/a25767e7-609c-42a7-8b8b-f793ee69d50f-kube-api-access-7kkj6\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/676d07a9-8585-4002-864f-d9c0d0d1e336-cni-binary-copy\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003544 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-cni-multus\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003571 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003597 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-cni-netd\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003631 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-var-lib-cni-multus\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003644 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-daemon-config\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003639 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-modprobe-d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003746 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a25767e7-609c-42a7-8b8b-f793ee69d50f-hosts-file\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003773 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-modprobe-d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003785 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-iptables-alerter-script\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.012760 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003825 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003850 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003871 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-sys-fs\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003912 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-os-release\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003949 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-etc-kubernetes\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003979 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-run\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.003999 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-sys-fs\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004003 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/014ceecb-1938-4660-b2fa-7c17bb94b782-host\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004046 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/014ceecb-1938-4660-b2fa-7c17bb94b782-serviceca\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004066 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-os-release\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004068 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-etc-kubernetes\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhmxl\" (UniqueName: \"kubernetes.io/projected/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-kube-api-access-qhmxl\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004100 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/676d07a9-8585-4002-864f-d9c0d0d1e336-cni-binary-copy\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004113 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-run\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004119 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/014ceecb-1938-4660-b2fa-7c17bb94b782-host\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.013426 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004154 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-system-cni-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004211 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-ovn\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004260 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-system-cni-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004271 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-log-socket\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004311 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-sys\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004348 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-lib-modules\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004371 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-host\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004400 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-host-slash\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004402 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004437 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-etc-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004464 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-host\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.014095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004493 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-conf-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004550 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-lib-modules\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004574 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-conf-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004602 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kds2d\" (UniqueName: \"kubernetes.io/projected/3ebeaae4-97b7-459a-9922-1435ab68b1a9-kube-api-access-kds2d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004603 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004617 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-sys\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004672 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/014ceecb-1938-4660-b2fa-7c17bb94b782-serviceca\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004726 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-cni-binary-copy\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004787 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4879c2c5-4dae-48c7-85b5-f010d8607f1e-agent-certs\") pod \"konnectivity-agent-s65dx\" (UID: \"4879c2c5-4dae-48c7-85b5-f010d8607f1e\") " pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004821 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-k8s-cni-cncf-io\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004853 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d1dde903-65f8-4917-97c5-6397a9f48c75-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhkmj\" (UniqueName: \"kubernetes.io/projected/676d07a9-8585-4002-864f-d9c0d0d1e336-kube-api-access-lhkmj\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.014765 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004895 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-tuned\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004908 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-k8s-cni-cncf-io\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004937 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebeaae4-97b7-459a-9922-1435ab68b1a9-tmp\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.004984 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-cni-bin\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005030 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-cni-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005068 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-netns\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005110 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-host-run-netns\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005139 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz7t6\" (UniqueName: \"kubernetes.io/projected/e7646b7b-d22f-4fab-9609-1c3beb2632e1-kube-api-access-mz7t6\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-node-log\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005193 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-ovnkube-config\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005222 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-socket-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005235 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/676d07a9-8585-4002-864f-d9c0d0d1e336-multus-cni-dir\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005249 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-device-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005249 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005289 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-device-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.015447 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005343 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8a8ceb14-f428-4d2d-ac73-5c1a777be296-socket-dir\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005360 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysctl-conf\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005389 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-kubelet\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.005490 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-sysctl-conf\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.007823 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.007842 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.007854 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rmpvb for pod openshift-network-diagnostics/network-check-target-6f44w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.007935 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb podName:d5019923-214b-4e9b-a0fd-2f76937c3ea7 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:34.507905167 +0000 UTC m=+1.913438076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rmpvb" (UniqueName: "kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb") pod "network-check-target-6f44w" (UID: "d5019923-214b-4e9b-a0fd-2f76937c3ea7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.008348 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3ebeaae4-97b7-459a-9922-1435ab68b1a9-etc-tuned\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.008616 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ebeaae4-97b7-459a-9922-1435ab68b1a9-tmp\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.010724 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-546lp\" (UniqueName: \"kubernetes.io/projected/014ceecb-1938-4660-b2fa-7c17bb94b782-kube-api-access-546lp\") pod \"node-ca-th6lw\" (UID: \"014ceecb-1938-4660-b2fa-7c17bb94b782\") " pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.011004 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcs4v\" (UniqueName: \"kubernetes.io/projected/d1dde903-65f8-4917-97c5-6397a9f48c75-kube-api-access-fcs4v\") pod \"multus-additional-cni-plugins-4bqh7\" (UID: \"d1dde903-65f8-4917-97c5-6397a9f48c75\") " pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.011638 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwmsx\" (UniqueName: \"kubernetes.io/projected/8a8ceb14-f428-4d2d-ac73-5c1a777be296-kube-api-access-gwmsx\") pod \"aws-ebs-csi-driver-node-5nhkq\" (UID: \"8a8ceb14-f428-4d2d-ac73-5c1a777be296\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.012470 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhkmj\" (UniqueName: \"kubernetes.io/projected/676d07a9-8585-4002-864f-d9c0d0d1e336-kube-api-access-lhkmj\") pod \"multus-bzvd4\" (UID: \"676d07a9-8585-4002-864f-d9c0d0d1e336\") " pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.016123 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.012684 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kds2d\" (UniqueName: \"kubernetes.io/projected/3ebeaae4-97b7-459a-9922-1435ab68b1a9-kube-api-access-kds2d\") pod \"tuned-vwrlp\" (UID: \"3ebeaae4-97b7-459a-9922-1435ab68b1a9\") " pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.027005 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.053760 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-th6lw" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.070727 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106358 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f5de882-ee4b-4c48-bd5e-87726ab35b72-kubelet-config\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106384 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f5de882-ee4b-4c48-bd5e-87726ab35b72-dbus\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-slash\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106471 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-slash\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106483 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f5de882-ee4b-4c48-bd5e-87726ab35b72-dbus\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a25767e7-609c-42a7-8b8b-f793ee69d50f-tmp-dir\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106541 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-ovnkube-script-lib\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106560 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4879c2c5-4dae-48c7-85b5-f010d8607f1e-konnectivity-ca\") pod \"konnectivity-agent-s65dx\" (UID: \"4879c2c5-4dae-48c7-85b5-f010d8607f1e\") " pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106562 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f5de882-ee4b-4c48-bd5e-87726ab35b72-kubelet-config\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106577 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kkj6\" (UniqueName: \"kubernetes.io/projected/a25767e7-609c-42a7-8b8b-f793ee69d50f-kube-api-access-7kkj6\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106607 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106639 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-cni-netd\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.136075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a25767e7-609c-42a7-8b8b-f793ee69d50f-hosts-file\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106673 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-iptables-alerter-script\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106720 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-cni-netd\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106727 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106747 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhmxl\" (UniqueName: \"kubernetes.io/projected/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-kube-api-access-qhmxl\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106777 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a25767e7-609c-42a7-8b8b-f793ee69d50f-hosts-file\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106788 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106825 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-ovn\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106849 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-log-socket\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.106861 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106875 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-host-slash\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106898 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-etc-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106922 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106969 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.137771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.106973 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-log-socket\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.106982 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret podName:6f5de882-ee4b-4c48-bd5e-87726ab35b72 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:34.606962165 +0000 UTC m=+2.012495075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret") pod "global-pull-secret-syncer-vtk7l" (UID: "6f5de882-ee4b-4c48-bd5e-87726ab35b72") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107011 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-host-slash\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107014 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-etc-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107051 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-ovn\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107050 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a25767e7-609c-42a7-8b8b-f793ee69d50f-tmp-dir\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107079 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107091 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-run-ovn-kubernetes\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4879c2c5-4dae-48c7-85b5-f010d8607f1e-agent-certs\") pod \"konnectivity-agent-s65dx\" (UID: \"4879c2c5-4dae-48c7-85b5-f010d8607f1e\") " pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107151 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-cni-bin\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.107159 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107181 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7t6\" (UniqueName: \"kubernetes.io/projected/e7646b7b-d22f-4fab-9609-1c3beb2632e1-kube-api-access-mz7t6\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107206 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-node-log\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107229 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-ovnkube-script-lib\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.138742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107240 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4879c2c5-4dae-48c7-85b5-f010d8607f1e-konnectivity-ca\") pod \"konnectivity-agent-s65dx\" (UID: \"4879c2c5-4dae-48c7-85b5-f010d8607f1e\") " pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107251 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-node-log\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.107239 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:34.607221974 +0000 UTC m=+2.012754887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107279 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-ovnkube-config\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107285 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-cni-bin\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107318 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-kubelet\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107348 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-systemd-units\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107375 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-run-netns\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107421 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-run-netns\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107429 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-host-kubelet\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107459 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-systemd-units\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107465 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64899c8b-c773-4375-a064-80b0de31b920-ovn-node-metrics-cert\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107537 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-var-lib-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107559 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-env-overrides\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.139914 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107577 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6bpk\" (UniqueName: \"kubernetes.io/projected/64899c8b-c773-4375-a064-80b0de31b920-kube-api-access-p6bpk\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107598 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-systemd\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107668 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-var-lib-openvswitch\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107716 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64899c8b-c773-4375-a064-80b0de31b920-run-systemd\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107856 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-ovnkube-config\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.107872 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-iptables-alerter-script\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.108196 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64899c8b-c773-4375-a064-80b0de31b920-env-overrides\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.110062 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64899c8b-c773-4375-a064-80b0de31b920-ovn-node-metrics-cert\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.110067 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4879c2c5-4dae-48c7-85b5-f010d8607f1e-agent-certs\") pod \"konnectivity-agent-s65dx\" (UID: \"4879c2c5-4dae-48c7-85b5-f010d8607f1e\") " pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.115725 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kkj6\" (UniqueName: \"kubernetes.io/projected/a25767e7-609c-42a7-8b8b-f793ee69d50f-kube-api-access-7kkj6\") pod \"node-resolver-vx2ng\" (UID: \"a25767e7-609c-42a7-8b8b-f793ee69d50f\") " pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.115793 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhmxl\" (UniqueName: \"kubernetes.io/projected/c3af89e5-d752-45cf-adb9-fa3ba9af50cd-kube-api-access-qhmxl\") pod \"iptables-alerter-mfvhl\" (UID: \"c3af89e5-d752-45cf-adb9-fa3ba9af50cd\") " pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.115802 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7t6\" (UniqueName: \"kubernetes.io/projected/e7646b7b-d22f-4fab-9609-1c3beb2632e1-kube-api-access-mz7t6\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.116022 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6bpk\" (UniqueName: \"kubernetes.io/projected/64899c8b-c773-4375-a064-80b0de31b920-kube-api-access-p6bpk\") pod \"ovnkube-node-z4cwb\" (UID: \"64899c8b-c773-4375-a064-80b0de31b920\") " pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.140844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.132638 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vx2ng" Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.161830 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mfvhl" Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.198405 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3af89e5_d752_45cf_adb9_fa3ba9af50cd.slice/crio-32d157e5316d6d992d6c6b5829e326454a46c9970ab26e3b02bbb56c73301979 WatchSource:0}: Error finding container 32d157e5316d6d992d6c6b5829e326454a46c9970ab26e3b02bbb56c73301979: Status 404 returned error can't find the container with id 32d157e5316d6d992d6c6b5829e326454a46c9970ab26e3b02bbb56c73301979 Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.199074 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod548ab702b5d1e4dcb62c99874dd47f27.slice/crio-d8f0761e2f82b3e7caaa9b1fe60618fcb1414b6206dd7a073e74291b839a5909 WatchSource:0}: Error finding container d8f0761e2f82b3e7caaa9b1fe60618fcb1414b6206dd7a073e74291b839a5909: Status 404 returned error can't find the container with id d8f0761e2f82b3e7caaa9b1fe60618fcb1414b6206dd7a073e74291b839a5909 Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.208056 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe29e2175c9c75acb15d1d4b57872d2.slice/crio-2f52b44192b8eb2a1fa5e904112a10f9b264448337e888476900c4748e45e8a3 WatchSource:0}: Error finding container 2f52b44192b8eb2a1fa5e904112a10f9b264448337e888476900c4748e45e8a3: Status 404 returned error can't find the container with id 2f52b44192b8eb2a1fa5e904112a10f9b264448337e888476900c4748e45e8a3 Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.208178 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.228570 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.238748 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.239212 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1dde903_65f8_4917_97c5_6397a9f48c75.slice/crio-49736cfddc161fe651ea0196a6e68ae3f299d3eb193995281014ed984cdb278b WatchSource:0}: Error finding container 49736cfddc161fe651ea0196a6e68ae3f299d3eb193995281014ed984cdb278b: Status 404 returned error can't find the container with id 49736cfddc161fe651ea0196a6e68ae3f299d3eb193995281014ed984cdb278b Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.271613 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.280940 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8ceb14_f428_4d2d_ac73_5c1a777be296.slice/crio-22f45a0d634aabaf5f82ca5601ddb7203d0fb62597a42c8d6ddd852db8a9f8ab WatchSource:0}: Error finding container 22f45a0d634aabaf5f82ca5601ddb7203d0fb62597a42c8d6ddd852db8a9f8ab: Status 404 returned error can't find the container with id 22f45a0d634aabaf5f82ca5601ddb7203d0fb62597a42c8d6ddd852db8a9f8ab Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.282645 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bzvd4" Jul 06 17:40:34.318120 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.291205 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676d07a9_8585_4002_864f_d9c0d0d1e336.slice/crio-fb6e28e687ce3088c58374b28146975c4f8de49353a8664feeaf3bcdb4a51f30 WatchSource:0}: Error finding container fb6e28e687ce3088c58374b28146975c4f8de49353a8664feeaf3bcdb4a51f30: Status 404 returned error can't find the container with id fb6e28e687ce3088c58374b28146975c4f8de49353a8664feeaf3bcdb4a51f30 Jul 06 17:40:34.458017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.370624 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:40:34.458017 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.383813 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64899c8b_c773_4375_a064_80b0de31b920.slice/crio-6b361b3672ee0f1c0ab09b072c9c17925b027f6290a36b5054e4653bffa817c5 WatchSource:0}: Error finding container 6b361b3672ee0f1c0ab09b072c9c17925b027f6290a36b5054e4653bffa817c5: Status 404 returned error can't find the container with id 6b361b3672ee0f1c0ab09b072c9c17925b027f6290a36b5054e4653bffa817c5 Jul 06 17:40:34.458017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.395360 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:34.458017 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.404358 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4879c2c5_4dae_48c7_85b5_f010d8607f1e.slice/crio-e6f78a7f4e9a820a6e4822b80d3e73ef076300dd215d389726329916bcc77824 WatchSource:0}: Error finding container e6f78a7f4e9a820a6e4822b80d3e73ef076300dd215d389726329916bcc77824: Status 404 returned error can't find the container with id e6f78a7f4e9a820a6e4822b80d3e73ef076300dd215d389726329916bcc77824 Jul 06 17:40:34.458017 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.426826 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebeaae4_97b7_459a_9922_1435ab68b1a9.slice/crio-b8b5cc5b166d2d4ae2ae267997afc670e78183cabe9383ef0f920bce9c88a658 WatchSource:0}: Error finding container b8b5cc5b166d2d4ae2ae267997afc670e78183cabe9383ef0f920bce9c88a658: Status 404 returned error can't find the container with id b8b5cc5b166d2d4ae2ae267997afc670e78183cabe9383ef0f920bce9c88a658 Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.483310 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25767e7_609c_42a7_8b8b_f793ee69d50f.slice/crio-28f83f532a8b7a2dd46be2652d7e58b33fcd895877dcea4dce44370cc2db5257 WatchSource:0}: Error finding container 28f83f532a8b7a2dd46be2652d7e58b33fcd895877dcea4dce44370cc2db5257: Status 404 returned error can't find the container with id 28f83f532a8b7a2dd46be2652d7e58b33fcd895877dcea4dce44370cc2db5257 Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:40:34.484173 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod014ceecb_1938_4660_b2fa_7c17bb94b782.slice/crio-3d8d0a9bdad7e29bb831ec449dc8a9feacfcbf450aedf23aa6fa7735589746a5 WatchSource:0}: Error finding container 3d8d0a9bdad7e29bb831ec449dc8a9feacfcbf450aedf23aa6fa7735589746a5: Status 404 returned error can't find the container with id 3d8d0a9bdad7e29bb831ec449dc8a9feacfcbf450aedf23aa6fa7735589746a5 Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.511141 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.511241 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.511269 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.511281 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rmpvb for pod openshift-network-diagnostics/network-check-target-6f44w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.511325 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb podName:d5019923-214b-4e9b-a0fd-2f76937c3ea7 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:35.511312049 +0000 UTC m=+2.916844954 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmpvb" (UniqueName: "kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb") pod "network-check-target-6f44w" (UID: "d5019923-214b-4e9b-a0fd-2f76937c3ea7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.611773 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.611813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.611947 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.611966 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:34.642081 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.612000 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:35.61198645 +0000 UTC m=+3.017519356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:34.643643 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:34.612025 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret podName:6f5de882-ee4b-4c48-bd5e-87726ab35b72 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:35.612011487 +0000 UTC m=+3.017544392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret") pod "global-pull-secret-syncer-vtk7l" (UID: "6f5de882-ee4b-4c48-bd5e-87726ab35b72") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:34.943770 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.938556 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 17:35:34 +0000 UTC" deadline="2028-01-30 21:07:25.013204187 +0000 UTC" Jul 06 17:40:34.943770 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.938620 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13755h26m50.074588873s" Jul 06 17:40:34.980440 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:34.980169 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 17:40:35.068188 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.067261 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:35.068188 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.067408 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:35.068188 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.067878 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:35.068188 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.067985 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:35.083868 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.083755 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" event={"ID":"548ab702b5d1e4dcb62c99874dd47f27","Type":"ContainerStarted","Data":"d8f0761e2f82b3e7caaa9b1fe60618fcb1414b6206dd7a073e74291b839a5909"} Jul 06 17:40:35.107631 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.107523 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mfvhl" event={"ID":"c3af89e5-d752-45cf-adb9-fa3ba9af50cd","Type":"ContainerStarted","Data":"32d157e5316d6d992d6c6b5829e326454a46c9970ab26e3b02bbb56c73301979"} Jul 06 17:40:35.123826 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.123729 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vx2ng" event={"ID":"a25767e7-609c-42a7-8b8b-f793ee69d50f","Type":"ContainerStarted","Data":"28f83f532a8b7a2dd46be2652d7e58b33fcd895877dcea4dce44370cc2db5257"} Jul 06 17:40:35.129637 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.129556 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" event={"ID":"3ebeaae4-97b7-459a-9922-1435ab68b1a9","Type":"ContainerStarted","Data":"b8b5cc5b166d2d4ae2ae267997afc670e78183cabe9383ef0f920bce9c88a658"} Jul 06 17:40:35.140846 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.140727 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"6b361b3672ee0f1c0ab09b072c9c17925b027f6290a36b5054e4653bffa817c5"} Jul 06 17:40:35.159143 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.159107 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" event={"ID":"8a8ceb14-f428-4d2d-ac73-5c1a777be296","Type":"ContainerStarted","Data":"22f45a0d634aabaf5f82ca5601ddb7203d0fb62597a42c8d6ddd852db8a9f8ab"} Jul 06 17:40:35.171728 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.171592 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-th6lw" event={"ID":"014ceecb-1938-4660-b2fa-7c17bb94b782","Type":"ContainerStarted","Data":"3d8d0a9bdad7e29bb831ec449dc8a9feacfcbf450aedf23aa6fa7735589746a5"} Jul 06 17:40:35.195728 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.195610 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s65dx" event={"ID":"4879c2c5-4dae-48c7-85b5-f010d8607f1e","Type":"ContainerStarted","Data":"e6f78a7f4e9a820a6e4822b80d3e73ef076300dd215d389726329916bcc77824"} Jul 06 17:40:35.204817 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.204717 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bzvd4" event={"ID":"676d07a9-8585-4002-864f-d9c0d0d1e336","Type":"ContainerStarted","Data":"fb6e28e687ce3088c58374b28146975c4f8de49353a8664feeaf3bcdb4a51f30"} Jul 06 17:40:35.219009 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.218567 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerStarted","Data":"49736cfddc161fe651ea0196a6e68ae3f299d3eb193995281014ed984cdb278b"} Jul 06 17:40:35.244719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.244621 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" event={"ID":"2fe29e2175c9c75acb15d1d4b57872d2","Type":"ContainerStarted","Data":"2f52b44192b8eb2a1fa5e904112a10f9b264448337e888476900c4748e45e8a3"} Jul 06 17:40:35.521820 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.521773 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:35.522266 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.522029 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:40:35.522266 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.522055 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:40:35.522266 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.522068 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rmpvb for pod openshift-network-diagnostics/network-check-target-6f44w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:35.522266 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.522128 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb podName:d5019923-214b-4e9b-a0fd-2f76937c3ea7 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:37.522109653 +0000 UTC m=+4.927642564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmpvb" (UniqueName: "kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb") pod "network-check-target-6f44w" (UID: "d5019923-214b-4e9b-a0fd-2f76937c3ea7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:35.622657 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.622411 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:35.622657 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.622488 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:35.622657 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.622635 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:35.622952 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.622698 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret podName:6f5de882-ee4b-4c48-bd5e-87726ab35b72 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:37.62268087 +0000 UTC m=+5.028213797 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret") pod "global-pull-secret-syncer-vtk7l" (UID: "6f5de882-ee4b-4c48-bd5e-87726ab35b72") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:35.623027 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.622635 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:35.623099 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:35.623075 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:37.623055215 +0000 UTC m=+5.028588136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:35.939315 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.939076 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 17:35:34 +0000 UTC" deadline="2028-04-14 19:23:39.539818558 +0000 UTC" Jul 06 17:40:35.939315 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:35.939135 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15553h43m3.600686985s" Jul 06 17:40:36.065452 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:36.064931 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:36.065452 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:36.065078 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:37.065114 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:37.065078 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:37.065756 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:37.065078 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:37.065756 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.065238 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:37.065756 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.065336 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:37.539733 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:37.539690 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:37.539976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.539860 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:40:37.539976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.539900 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:40:37.539976 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.539915 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rmpvb for pod openshift-network-diagnostics/network-check-target-6f44w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:37.540151 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.540001 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb podName:d5019923-214b-4e9b-a0fd-2f76937c3ea7 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:41.539977963 +0000 UTC m=+8.945510896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmpvb" (UniqueName: "kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb") pod "network-check-target-6f44w" (UID: "d5019923-214b-4e9b-a0fd-2f76937c3ea7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:37.640653 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:37.640611 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:37.640832 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:37.640672 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:37.640832 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.640797 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:37.640980 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.640841 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:37.640980 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.640878 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret podName:6f5de882-ee4b-4c48-bd5e-87726ab35b72 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:41.64085784 +0000 UTC m=+9.046390762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret") pod "global-pull-secret-syncer-vtk7l" (UID: "6f5de882-ee4b-4c48-bd5e-87726ab35b72") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:37.640980 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:37.640901 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:41.640889169 +0000 UTC m=+9.046422089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:38.065997 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:38.065364 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:38.065997 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:38.065525 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:39.064956 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:39.064917 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:39.065136 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:39.065079 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:39.065752 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:39.064918 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:39.065752 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:39.065709 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:40.064719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:40.064683 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:40.065230 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:40.064832 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:41.065640 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:41.065423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:41.065640 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:41.065480 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:41.065640 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.065617 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:41.066149 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.065987 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:41.578221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:41.578179 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:41.578549 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.578350 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:40:41.578549 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.578375 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:40:41.578549 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.578388 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rmpvb for pod openshift-network-diagnostics/network-check-target-6f44w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:41.578549 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.578452 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb podName:d5019923-214b-4e9b-a0fd-2f76937c3ea7 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:49.578432489 +0000 UTC m=+16.983965419 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmpvb" (UniqueName: "kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb") pod "network-check-target-6f44w" (UID: "d5019923-214b-4e9b-a0fd-2f76937c3ea7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:41.679161 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:41.678818 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:41.679161 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:41.678958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:41.679161 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.679011 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:41.679161 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.679082 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:41.679161 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.679097 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:49.679073905 +0000 UTC m=+17.084606827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:41.679161 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:41.679136 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret podName:6f5de882-ee4b-4c48-bd5e-87726ab35b72 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:49.679119403 +0000 UTC m=+17.084652327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret") pod "global-pull-secret-syncer-vtk7l" (UID: "6f5de882-ee4b-4c48-bd5e-87726ab35b72") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:42.065190 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:42.065150 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:42.065395 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:42.065312 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:43.065612 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:43.065578 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:43.066142 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:43.065722 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:43.066142 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:43.066055 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:43.066293 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:43.066140 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:44.064529 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:44.064485 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:44.064720 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:44.064630 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:45.065259 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:45.065224 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:45.065679 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:45.065372 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:45.065679 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:45.065447 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:45.065679 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:45.065560 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:46.064416 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:46.064382 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:46.064584 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:46.064530 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:47.065075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:47.065040 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:47.065075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:47.065062 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:47.065530 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:47.065192 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:47.065530 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:47.065329 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:48.064907 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:48.064868 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:48.065096 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:48.064984 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:49.065562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:49.065519 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:49.066033 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.065669 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:49.066087 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:49.065532 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:49.066178 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.066143 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:49.641031 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:49.640995 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:49.641211 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.641177 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:40:49.641211 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.641198 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:40:49.641211 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.641208 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rmpvb for pod openshift-network-diagnostics/network-check-target-6f44w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:49.641303 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.641263 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb podName:d5019923-214b-4e9b-a0fd-2f76937c3ea7 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:05.641246105 +0000 UTC m=+33.046779021 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmpvb" (UniqueName: "kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb") pod "network-check-target-6f44w" (UID: "d5019923-214b-4e9b-a0fd-2f76937c3ea7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:40:49.742340 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:49.742307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:49.742340 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:49.742353 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:49.742593 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.742481 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:49.742593 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.742550 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:05.74253311 +0000 UTC m=+33.148066035 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:40:49.742593 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.742481 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:49.742593 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:49.742584 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret podName:6f5de882-ee4b-4c48-bd5e-87726ab35b72 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:05.742577498 +0000 UTC m=+33.148110401 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret") pod "global-pull-secret-syncer-vtk7l" (UID: "6f5de882-ee4b-4c48-bd5e-87726ab35b72") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:40:50.065371 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:50.065339 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:50.065543 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:50.065481 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:51.064488 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:51.064453 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:51.064936 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:51.064503 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:51.064936 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:51.064588 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:51.064936 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:51.064707 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:52.065471 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:52.065225 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:52.065957 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:52.065579 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:53.065536 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:53.065503 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:53.066024 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:53.065692 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:53.066024 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:53.065731 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:53.066024 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:53.065816 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:54.064792 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:54.064758 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:54.064968 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:54.064868 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:55.064744 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.064670 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:55.065086 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.064670 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:55.065086 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:55.064800 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:55.065086 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:55.064879 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:55.296874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.295787 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bzvd4" event={"ID":"676d07a9-8585-4002-864f-d9c0d0d1e336","Type":"ContainerStarted","Data":"17c72cc3220e16baba5b4f4d8e40859310e547ed0af56aa317e02ddcea261125"} Jul 06 17:40:55.300680 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.300020 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" event={"ID":"2fe29e2175c9c75acb15d1d4b57872d2","Type":"ContainerStarted","Data":"e4e1c75cf199a76fe8431c13e5b2a74b53246e7e9046673b4ba142df55fa054e"} Jul 06 17:40:55.306764 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.306734 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" event={"ID":"3ebeaae4-97b7-459a-9922-1435ab68b1a9","Type":"ContainerStarted","Data":"14e7d1d96f9f76f3589e4e5c6f1f266acd77081837b3760b0718b0fa4cda9890"} Jul 06 17:40:55.308619 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.308597 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:40:55.309291 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.309159 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"e8b4d0da3824d16a22afcd305f9ff514c4eef4b9e082700b6758c0907d857056"} Jul 06 17:40:55.309291 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.309234 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"7b461c12bdbf66f377898c79b0eb2be85c834cf4d41105991671dcb3d193755b"} Jul 06 17:40:55.329697 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.329329 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bzvd4" podStartSLOduration=1.4199562989999999 podStartE2EDuration="22.329305216s" totalImagesPullingTime="20.909348917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:55.313681114 +0000 UTC m=+22.719214039" watchObservedRunningTime="2026-07-06 17:40:55.329305216 +0000 UTC m=+22.734838145" Jul 06 17:40:55.348318 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.348262 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vwrlp" podStartSLOduration=0.800423575 podStartE2EDuration="21.348239556s" totalImagesPullingTime="20.547815981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:55.348188199 +0000 UTC m=+22.753721124" watchObservedRunningTime="2026-07-06 17:40:55.348239556 +0000 UTC m=+22.753772493" Jul 06 17:40:55.349176 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:55.349131 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-115.ec2.internal" podStartSLOduration=21.349115231 podStartE2EDuration="21.349115231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:55.329810948 +0000 UTC m=+22.735343869" watchObservedRunningTime="2026-07-06 17:40:55.349115231 +0000 UTC m=+22.754648157" Jul 06 17:40:56.064445 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.064390 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:56.064603 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:56.064552 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:56.315683 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.312734 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-th6lw" event={"ID":"014ceecb-1938-4660-b2fa-7c17bb94b782","Type":"ContainerStarted","Data":"bdcf9b681adabf7a6c1e1ed4490ca214c90fa43b80d1fe09e58a44f6aa958212"} Jul 06 17:40:56.317014 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.316988 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s65dx" event={"ID":"4879c2c5-4dae-48c7-85b5-f010d8607f1e","Type":"ContainerStarted","Data":"0ff013faab6ea463e7a2c1ce13337244f24f809b27ab433b8b09af029743268e"} Jul 06 17:40:56.318391 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.318367 2558 generic.go:356] "Generic (PLEG): container finished" podID="d1dde903-65f8-4917-97c5-6397a9f48c75" containerID="ed2ba2f08490a791bad3a0329807fbe533c9c2be4c4487dfbb3a910c71e5c3ea" exitCode=0 Jul 06 17:40:56.318520 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.318444 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerDied","Data":"ed2ba2f08490a791bad3a0329807fbe533c9c2be4c4487dfbb3a910c71e5c3ea"} Jul 06 17:40:56.319999 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.319966 2558 generic.go:356] "Generic (PLEG): container finished" podID="548ab702b5d1e4dcb62c99874dd47f27" containerID="6b97b4ab7b62f191267512c1828f8b58b3c22bf453aeda2394217ae59eeab6dd" exitCode=0 Jul 06 17:40:56.320069 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.320033 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" event={"ID":"548ab702b5d1e4dcb62c99874dd47f27","Type":"ContainerDied","Data":"6b97b4ab7b62f191267512c1828f8b58b3c22bf453aeda2394217ae59eeab6dd"} Jul 06 17:40:56.321724 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.321527 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mfvhl" event={"ID":"c3af89e5-d752-45cf-adb9-fa3ba9af50cd","Type":"ContainerStarted","Data":"aaa2f3ddbae78ee829e6549dd5de702e023852b68c46ee3da0b1745c95a94f13"} Jul 06 17:40:56.322941 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.322917 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vx2ng" event={"ID":"a25767e7-609c-42a7-8b8b-f793ee69d50f","Type":"ContainerStarted","Data":"e2361d69b63ddc7eba7f858a214262a48d5c51269f659eb6aafd714c33af0380"} Jul 06 17:40:56.325554 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.325536 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:40:56.325838 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.325823 2558 generic.go:356] "Generic (PLEG): container finished" podID="64899c8b-c773-4375-a064-80b0de31b920" containerID="e8b4d0da3824d16a22afcd305f9ff514c4eef4b9e082700b6758c0907d857056" exitCode=1 Jul 06 17:40:56.325925 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.325889 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerDied","Data":"e8b4d0da3824d16a22afcd305f9ff514c4eef4b9e082700b6758c0907d857056"} Jul 06 17:40:56.325981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.325964 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"6800f4c606babd957b6410565b0ca5ec0891d274c4acbf2accfd0f4e11e2d046"} Jul 06 17:40:56.326034 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.326008 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"bc50483b78020f42e7cd2f2a04f26060977232b5edff47caea2bc4e48b0157e6"} Jul 06 17:40:56.326084 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.326069 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"0f1b520bfef331ed9325cdd094a3bf5d9167cd3566a2191d8791e9075f1a785f"} Jul 06 17:40:56.326124 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.326106 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"cff3a207837eb8366c5ba6a1456c4c1f165115ade44fe1e728fc0e3877b37564"} Jul 06 17:40:56.327261 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.327240 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" event={"ID":"8a8ceb14-f428-4d2d-ac73-5c1a777be296","Type":"ContainerStarted","Data":"8dcf61354b94f43ace4202262cc37db05f5d92a8a8c9d7dd81dba52bf8588a4e"} Jul 06 17:40:56.329213 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.329181 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-th6lw" podStartSLOduration=1.912382811 podStartE2EDuration="22.329169807s" totalImagesPullingTime="20.416786996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:56.328703658 +0000 UTC m=+23.734236581" watchObservedRunningTime="2026-07-06 17:40:56.329169807 +0000 UTC m=+23.734702742" Jul 06 17:40:56.341866 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.341816 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vx2ng" podStartSLOduration=1.8495610820000001 podStartE2EDuration="22.341805704s" totalImagesPullingTime="20.492244622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:56.341621424 +0000 UTC m=+23.747154348" watchObservedRunningTime="2026-07-06 17:40:56.341805704 +0000 UTC m=+23.747338627" Jul 06 17:40:56.371418 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.371369 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s65dx" podStartSLOduration=1.870185109 podStartE2EDuration="22.371351782s" totalImagesPullingTime="20.501166673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:56.370928971 +0000 UTC m=+23.776461898" watchObservedRunningTime="2026-07-06 17:40:56.371351782 +0000 UTC m=+23.776884708" Jul 06 17:40:56.387256 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:56.387218 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mfvhl" podStartSLOduration=1.687880135 podStartE2EDuration="22.387203194s" totalImagesPullingTime="20.699323059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:56.387028685 +0000 UTC m=+23.792561612" watchObservedRunningTime="2026-07-06 17:40:56.387203194 +0000 UTC m=+23.792736121" Jul 06 17:40:57.064906 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.064876 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:57.065217 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.064876 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:57.065217 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:57.065079 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:57.065331 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:57.065220 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:57.331269 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.331234 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" event={"ID":"548ab702b5d1e4dcb62c99874dd47f27","Type":"ContainerStarted","Data":"02eac01b08d0446788278469db4157c489ec34268a28a52f4de069c7e20e1e06"} Jul 06 17:40:57.344443 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.344383 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-115.ec2.internal" podStartSLOduration=23.344368984 podStartE2EDuration="23.344368984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:57.343970996 +0000 UTC m=+24.749503920" watchObservedRunningTime="2026-07-06 17:40:57.344368984 +0000 UTC m=+24.749901909" Jul 06 17:40:57.360645 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.360619 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 17:40:57.983966 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.983859 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T17:40:57.360639659Z","UUID":"b746b311-d12b-4251-878a-89eac3c1d75a","Handler":null,"Name":"","Endpoint":""} Jul 06 17:40:57.985871 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.985837 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 17:40:57.986008 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:57.985952 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 17:40:58.065233 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:58.065208 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:40:58.065397 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:58.065321 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:40:58.336032 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:58.336007 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:40:58.336408 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:58.336352 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"ef45c3c057b973898b399675c22bd2353d8ba8d64720d1487df509f40c47c630"} Jul 06 17:40:58.337989 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:58.337965 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" event={"ID":"8a8ceb14-f428-4d2d-ac73-5c1a777be296","Type":"ContainerStarted","Data":"fe122cdf0e4a5ab677c752db16475903389bed327c9e30069a83fba4cd9698cc"} Jul 06 17:40:58.548403 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:58.548361 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:58.548901 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:58.548884 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:59.065017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:59.064977 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:40:59.065202 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:59.065119 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:40:59.065202 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:59.065171 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:40:59.065318 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:40:59.065281 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:40:59.342519 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:59.342423 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" event={"ID":"8a8ceb14-f428-4d2d-ac73-5c1a777be296","Type":"ContainerStarted","Data":"576fa405b35121d9ccf346ace8f64ea19928cf7f82194d81cf0723f450ce1bda"} Jul 06 17:40:59.342867 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:59.342649 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:59.343147 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:59.343126 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s65dx" Jul 06 17:40:59.357295 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:40:59.357245 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5nhkq" podStartSLOduration=3.69871341 podStartE2EDuration="26.357233462s" totalImagesPullingTime="22.658520052s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:59.356922055 +0000 UTC m=+26.762454979" watchObservedRunningTime="2026-07-06 17:40:59.357233462 +0000 UTC m=+26.762766386" Jul 06 17:41:00.065369 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:00.065346 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:00.065502 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:00.065471 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:41:00.348271 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:00.348202 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:41:00.348926 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:00.348608 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"edcc26e56e1acbcd4cd757bac85ddff97f6e30a0061b41706d54028701c23fb8"} Jul 06 17:41:00.349176 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:00.349151 2558 scope.go:117] "RemoveContainer" containerID="e8b4d0da3824d16a22afcd305f9ff514c4eef4b9e082700b6758c0907d857056" Jul 06 17:41:01.065413 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:01.065376 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:01.065598 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:01.065527 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:41:01.065598 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:01.065539 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:01.065706 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:01.065631 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:41:02.064932 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.064747 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:02.065590 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:02.065031 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:41:02.065879 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.065846 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6f44w"] Jul 06 17:41:02.065982 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.065966 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:02.066094 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:02.066071 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:41:02.069721 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.069695 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vtk7l"] Jul 06 17:41:02.070390 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.070370 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b9gd9"] Jul 06 17:41:02.070512 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.070481 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:02.070613 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:02.070589 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:41:02.355296 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.355259 2558 generic.go:356] "Generic (PLEG): container finished" podID="d1dde903-65f8-4917-97c5-6397a9f48c75" containerID="308b6a3445d3fccba3ecd0e91c2c4e4362f4cead18adc81149e33d2d74a47726" exitCode=0 Jul 06 17:41:02.355489 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.355342 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerDied","Data":"308b6a3445d3fccba3ecd0e91c2c4e4362f4cead18adc81149e33d2d74a47726"} Jul 06 17:41:02.359352 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.359335 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:41:02.359707 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.359674 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" event={"ID":"64899c8b-c773-4375-a064-80b0de31b920","Type":"ContainerStarted","Data":"d1685a521dbd8944b0a6745e7b022fa246ceafa0195280e33ac051588dc0b1d0"} Jul 06 17:41:02.359798 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.359681 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:02.359845 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:02.359828 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:41:02.359988 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.359969 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:41:02.360091 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.360029 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:41:02.360091 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.360068 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:41:02.376318 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.376294 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:41:02.376454 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.376383 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:41:02.401842 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:02.401794 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" podStartSLOduration=7.760620411 podStartE2EDuration="28.401774697s" totalImagesPullingTime="20.641154286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:02.40126507 +0000 UTC m=+29.806798007" watchObservedRunningTime="2026-07-06 17:41:02.401774697 +0000 UTC m=+29.807307622" Jul 06 17:41:04.065151 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:04.065120 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:04.065523 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:04.065124 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:04.065523 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:04.065240 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:41:04.065523 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:04.065292 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:41:04.065523 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:04.065120 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:04.065523 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:04.065379 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:41:04.366728 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:04.366649 2558 generic.go:356] "Generic (PLEG): container finished" podID="d1dde903-65f8-4917-97c5-6397a9f48c75" containerID="78eb1cfd7060d4fe65575e274ffac225859a4c80640522569f2370e295fb4159" exitCode=0 Jul 06 17:41:04.366862 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:04.366733 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerDied","Data":"78eb1cfd7060d4fe65575e274ffac225859a4c80640522569f2370e295fb4159"} Jul 06 17:41:05.662310 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:05.662217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:05.662681 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.662347 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 17:41:05.662681 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.662365 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 17:41:05.662681 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.662375 2558 projected.go:196] Error preparing data for projected volume kube-api-access-rmpvb for pod openshift-network-diagnostics/network-check-target-6f44w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:41:05.662681 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.662443 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb podName:d5019923-214b-4e9b-a0fd-2f76937c3ea7 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:37.662411188 +0000 UTC m=+65.067944093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rmpvb" (UniqueName: "kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb") pod "network-check-target-6f44w" (UID: "d5019923-214b-4e9b-a0fd-2f76937c3ea7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 17:41:05.762611 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:05.762570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:05.762611 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:05.762613 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:05.762820 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.762715 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 17:41:05.762820 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.762781 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret podName:6f5de882-ee4b-4c48-bd5e-87726ab35b72 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:37.762758986 +0000 UTC m=+65.168291892 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret") pod "global-pull-secret-syncer-vtk7l" (UID: "6f5de882-ee4b-4c48-bd5e-87726ab35b72") : object "kube-system"/"original-pull-secret" not registered Jul 06 17:41:05.762820 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.762720 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:41:05.762951 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:05.762849 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:37.762837353 +0000 UTC m=+65.168370257 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 17:41:06.064387 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:06.064360 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:06.064555 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:06.064363 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:06.064555 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:06.064476 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:41:06.064555 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:06.064360 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:06.064656 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:06.064541 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:41:06.064656 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:06.064611 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:41:06.374121 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:06.374041 2558 generic.go:356] "Generic (PLEG): container finished" podID="d1dde903-65f8-4917-97c5-6397a9f48c75" containerID="4fb8d0de6a3824fe2914aac70d2315c09fa90b790f314a54c2a08e03da757823" exitCode=0 Jul 06 17:41:06.374121 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:06.374086 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerDied","Data":"4fb8d0de6a3824fe2914aac70d2315c09fa90b790f314a54c2a08e03da757823"} Jul 06 17:41:08.064949 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.064913 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:08.065380 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.064916 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:08.065380 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:08.065036 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6f44w" podUID="d5019923-214b-4e9b-a0fd-2f76937c3ea7" Jul 06 17:41:08.065380 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.064917 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:08.065380 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:08.065105 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtk7l" podUID="6f5de882-ee4b-4c48-bd5e-87726ab35b72" Jul 06 17:41:08.065380 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:08.065217 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:41:08.825607 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.825580 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-115.ec2.internal" event="NodeReady" Jul 06 17:41:08.825774 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.825719 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 17:41:08.863120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.863068 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mgxwp"] Jul 06 17:41:08.866756 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.866688 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jpln9"] Jul 06 17:41:08.866874 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.866832 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:08.869050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.869028 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 17:41:08.869050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.869040 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmt7g\"" Jul 06 17:41:08.869222 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.869163 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 17:41:08.870459 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.870439 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:08.872054 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.872038 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xxwrm\"" Jul 06 17:41:08.872245 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.872229 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 17:41:08.872327 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.872307 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 17:41:08.872473 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.872454 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 17:41:08.875746 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.875725 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mgxwp"] Jul 06 17:41:08.879892 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.879874 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jpln9"] Jul 06 17:41:08.984459 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.984412 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-config-volume\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:08.984675 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.984500 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:08.984675 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.984539 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-864nm\" (UniqueName: \"kubernetes.io/projected/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-kube-api-access-864nm\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:08.984675 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.984576 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-tmp-dir\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:08.984675 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.984654 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f78rk\" (UniqueName: \"kubernetes.io/projected/3a2ef3dd-ede8-498e-ae66-0941952e042a-kube-api-access-f78rk\") pod \"ingress-canary-jpln9\" (UID: \"3a2ef3dd-ede8-498e-ae66-0941952e042a\") " pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:08.984869 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:08.984727 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a2ef3dd-ede8-498e-ae66-0941952e042a-cert\") pod \"ingress-canary-jpln9\" (UID: \"3a2ef3dd-ede8-498e-ae66-0941952e042a\") " pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:09.085636 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.085608 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a2ef3dd-ede8-498e-ae66-0941952e042a-cert\") pod \"ingress-canary-jpln9\" (UID: \"3a2ef3dd-ede8-498e-ae66-0941952e042a\") " pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:09.086062 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.085650 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-config-volume\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.086062 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.085679 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.086062 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:09.085774 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:41:09.086062 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:09.085832 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:09.585814362 +0000 UTC m=+36.991347270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:41:09.086062 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.085921 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-864nm\" (UniqueName: \"kubernetes.io/projected/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-kube-api-access-864nm\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.086062 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.085969 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-tmp-dir\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.086062 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.086019 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rk\" (UniqueName: \"kubernetes.io/projected/3a2ef3dd-ede8-498e-ae66-0941952e042a-kube-api-access-f78rk\") pod \"ingress-canary-jpln9\" (UID: \"3a2ef3dd-ede8-498e-ae66-0941952e042a\") " pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:09.086376 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.086352 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-config-volume\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.086463 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.086445 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-tmp-dir\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.090580 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.090558 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a2ef3dd-ede8-498e-ae66-0941952e042a-cert\") pod \"ingress-canary-jpln9\" (UID: \"3a2ef3dd-ede8-498e-ae66-0941952e042a\") " pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:09.098109 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.098090 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-864nm\" (UniqueName: \"kubernetes.io/projected/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-kube-api-access-864nm\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.098371 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.098350 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f78rk\" (UniqueName: \"kubernetes.io/projected/3a2ef3dd-ede8-498e-ae66-0941952e042a-kube-api-access-f78rk\") pod \"ingress-canary-jpln9\" (UID: \"3a2ef3dd-ede8-498e-ae66-0941952e042a\") " pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:09.184938 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.184851 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jpln9" Jul 06 17:41:09.590281 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:09.590239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:09.590484 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:09.590405 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:41:09.590550 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:09.590490 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:10.590469271 +0000 UTC m=+37.996002176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:41:10.064860 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.064821 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:10.065052 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.064944 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:10.065119 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.065050 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:10.067803 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.067780 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gwzng\"" Jul 06 17:41:10.067803 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.067801 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 17:41:10.067981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.067868 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 17:41:10.067981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.067805 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 17:41:10.067981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.067872 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vsjgr\"" Jul 06 17:41:10.067981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.067807 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 17:41:10.487374 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.487203 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jpln9"] Jul 06 17:41:10.490471 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:41:10.490444 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2ef3dd_ede8_498e_ae66_0941952e042a.slice/crio-457a2c0a3a95901dec05e25d8791b9c4503404a128f807ea98598b5439f63fc1 WatchSource:0}: Error finding container 457a2c0a3a95901dec05e25d8791b9c4503404a128f807ea98598b5439f63fc1: Status 404 returned error can't find the container with id 457a2c0a3a95901dec05e25d8791b9c4503404a128f807ea98598b5439f63fc1 Jul 06 17:41:10.597555 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:10.597527 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:10.597715 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:10.597693 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:41:10.597799 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:10.597782 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:12.597761152 +0000 UTC m=+40.003294055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:41:11.388009 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:11.387960 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jpln9" event={"ID":"3a2ef3dd-ede8-498e-ae66-0941952e042a","Type":"ContainerStarted","Data":"457a2c0a3a95901dec05e25d8791b9c4503404a128f807ea98598b5439f63fc1"} Jul 06 17:41:11.390922 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:11.390890 2558 generic.go:356] "Generic (PLEG): container finished" podID="d1dde903-65f8-4917-97c5-6397a9f48c75" containerID="76d000e4f0742780e410c2611d1604400026199159c14360a9f9f79a7b064139" exitCode=0 Jul 06 17:41:11.391057 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:11.390939 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerDied","Data":"76d000e4f0742780e410c2611d1604400026199159c14360a9f9f79a7b064139"} Jul 06 17:41:12.397474 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:12.397439 2558 generic.go:356] "Generic (PLEG): container finished" podID="d1dde903-65f8-4917-97c5-6397a9f48c75" containerID="5aa82a94ff6f6c59031d23cc34cdc3deddc81226c23c5a6f09db64e74de6f399" exitCode=0 Jul 06 17:41:12.398003 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:12.397492 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerDied","Data":"5aa82a94ff6f6c59031d23cc34cdc3deddc81226c23c5a6f09db64e74de6f399"} Jul 06 17:41:12.616013 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:12.615970 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:12.616185 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:12.616143 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:41:12.616260 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:12.616219 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:16.61619988 +0000 UTC m=+44.021732785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:41:13.403305 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:13.403215 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" event={"ID":"d1dde903-65f8-4917-97c5-6397a9f48c75","Type":"ContainerStarted","Data":"e5dfa1360f3243a9e32710eca03afa4ceeda31c9d4b14ec7643391ec43beee8a"} Jul 06 17:41:13.404689 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:13.404658 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jpln9" event={"ID":"3a2ef3dd-ede8-498e-ae66-0941952e042a","Type":"ContainerStarted","Data":"5b492b55277900d93823ddbc168a22962e1dc7674ab230f413e0786184a1be57"} Jul 06 17:41:13.422945 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:13.422895 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4bqh7" podStartSLOduration=8.373108279 podStartE2EDuration="40.422881723s" totalImagesPullingTime="32.049773444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:13.422572478 +0000 UTC m=+40.828105395" watchObservedRunningTime="2026-07-06 17:41:13.422881723 +0000 UTC m=+40.828414650" Jul 06 17:41:13.437219 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:13.437167 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jpln9" podStartSLOduration=2.796548568 podStartE2EDuration="5.437148366s" totalImagesPullingTime="2.640599798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:13.436780874 +0000 UTC m=+40.842313799" watchObservedRunningTime="2026-07-06 17:41:13.437148366 +0000 UTC m=+40.842681287" Jul 06 17:41:16.644540 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:16.644501 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:16.644967 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:16.644659 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:41:16.644967 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:16.644722 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:24.644707667 +0000 UTC m=+52.050240570 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:41:24.698936 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:24.698892 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:24.699559 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:24.699077 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:41:24.699559 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:24.699184 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:40.699163616 +0000 UTC m=+68.104696528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:41:34.385956 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:34.385923 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z4cwb" Jul 06 17:41:37.694855 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.694806 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:37.696627 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.696606 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 17:41:37.707463 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.707424 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 17:41:37.720307 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.720274 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmpvb\" (UniqueName: \"kubernetes.io/projected/d5019923-214b-4e9b-a0fd-2f76937c3ea7-kube-api-access-rmpvb\") pod \"network-check-target-6f44w\" (UID: \"d5019923-214b-4e9b-a0fd-2f76937c3ea7\") " pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:37.795516 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.795474 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:37.795516 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.795520 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:41:37.798520 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.798502 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 17:41:37.798600 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.798503 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 17:41:37.806314 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:37.806289 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 17:41:37.806393 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:37.806354 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:42:41.806332975 +0000 UTC m=+129.211865881 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : secret "metrics-daemon-secret" not found Jul 06 17:41:37.808366 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.808347 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f5de882-ee4b-4c48-bd5e-87726ab35b72-original-pull-secret\") pod \"global-pull-secret-syncer-vtk7l\" (UID: \"6f5de882-ee4b-4c48-bd5e-87726ab35b72\") " pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:37.979365 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.979286 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vsjgr\"" Jul 06 17:41:37.987842 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.987822 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:37.993372 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:37.993354 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtk7l" Jul 06 17:41:38.128210 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:38.128181 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vtk7l"] Jul 06 17:41:38.131056 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:41:38.131026 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5de882_ee4b_4c48_bd5e_87726ab35b72.slice/crio-31a3552fc35e47c4981197fd5f2cfe795323bb2e8de736aa6187775d2ec814c7 WatchSource:0}: Error finding container 31a3552fc35e47c4981197fd5f2cfe795323bb2e8de736aa6187775d2ec814c7: Status 404 returned error can't find the container with id 31a3552fc35e47c4981197fd5f2cfe795323bb2e8de736aa6187775d2ec814c7 Jul 06 17:41:38.144072 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:38.144045 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6f44w"] Jul 06 17:41:38.147552 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:41:38.147530 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5019923_214b_4e9b_a0fd_2f76937c3ea7.slice/crio-89e656354ebf3484d438366db6f74b318f6f15f17cdb0021fa8b2cca42f12409 WatchSource:0}: Error finding container 89e656354ebf3484d438366db6f74b318f6f15f17cdb0021fa8b2cca42f12409: Status 404 returned error can't find the container with id 89e656354ebf3484d438366db6f74b318f6f15f17cdb0021fa8b2cca42f12409 Jul 06 17:41:38.460445 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:38.460397 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vtk7l" event={"ID":"6f5de882-ee4b-4c48-bd5e-87726ab35b72","Type":"ContainerStarted","Data":"31a3552fc35e47c4981197fd5f2cfe795323bb2e8de736aa6187775d2ec814c7"} Jul 06 17:41:38.461471 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:38.461449 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6f44w" event={"ID":"d5019923-214b-4e9b-a0fd-2f76937c3ea7","Type":"ContainerStarted","Data":"89e656354ebf3484d438366db6f74b318f6f15f17cdb0021fa8b2cca42f12409"} Jul 06 17:41:40.718721 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:40.718680 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:41:40.719198 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:40.718821 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:41:40.719198 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:41:40.718901 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:42:12.718879994 +0000 UTC m=+100.124412901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:41:43.477412 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:43.477372 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vtk7l" event={"ID":"6f5de882-ee4b-4c48-bd5e-87726ab35b72","Type":"ContainerStarted","Data":"1f17dfa66819c45bdaf0f06daaf69d66f278eb032afbaf25cc227a36a3934106"} Jul 06 17:41:43.478678 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:43.478656 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6f44w" event={"ID":"d5019923-214b-4e9b-a0fd-2f76937c3ea7","Type":"ContainerStarted","Data":"e3aced315f55a158e0c5ef888fe8f3a2385e41e3fb423ee00e0bfefcd4039f96"} Jul 06 17:41:43.478801 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:43.478785 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:41:43.491129 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:43.491077 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vtk7l" podStartSLOduration=64.620775995 podStartE2EDuration="1m9.491062255s" totalImagesPullingTime="4.87028626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:43.490766756 +0000 UTC m=+70.896299683" watchObservedRunningTime="2026-07-06 17:41:43.491062255 +0000 UTC m=+70.896595182" Jul 06 17:41:43.503124 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:41:43.503073 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6f44w" podStartSLOduration=65.660715011 podStartE2EDuration="1m10.503058917s" totalImagesPullingTime="4.842343906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:43.502963303 +0000 UTC m=+70.908496230" watchObservedRunningTime="2026-07-06 17:41:43.503058917 +0000 UTC m=+70.908591842" Jul 06 17:42:12.733232 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:42:12.733174 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:42:12.733705 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:42:12.733303 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:42:12.733705 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:42:12.733371 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:43:16.733357512 +0000 UTC m=+164.138890416 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:42:14.483583 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:42:14.483553 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6f44w" Jul 06 17:42:41.829653 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:42:41.829619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:42:41.830130 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:42:41.829723 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 17:42:41.830130 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:42:41.829777 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:43.829760171 +0000 UTC m=+251.235293076 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : secret "metrics-daemon-secret" not found Jul 06 17:43:11.879205 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:43:11.879111 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mgxwp" podUID="a34e0be6-2c80-4817-80dd-5c4fbc90feb1" Jul 06 17:43:12.690690 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:43:12.690657 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mgxwp" Jul 06 17:43:13.084596 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:43:13.084562 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:43:16.763226 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:43:16.763194 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:43:16.763725 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:43:16.763308 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 17:43:16.763725 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:43:16.763374 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls podName:a34e0be6-2c80-4817-80dd-5c4fbc90feb1 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:18.763362444 +0000 UTC m=+286.168895349 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls") pod "dns-default-mgxwp" (UID: "a34e0be6-2c80-4817-80dd-5c4fbc90feb1") : secret "dns-default-metrics-tls" not found Jul 06 17:43:27.068221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:43:27.068134 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:44:29.356644 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.356601 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m"] Jul 06 17:44:29.359732 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.359707 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cz4mf"] Jul 06 17:44:29.359891 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.359834 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.361863 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.361838 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 17:44:29.362286 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.362145 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 17:44:29.362826 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.362604 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 17:44:29.363638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.363613 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 17:44:29.363638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.363637 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ttxk6\"" Jul 06 17:44:29.364071 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.364052 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-wg42h"] Jul 06 17:44:29.364212 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.364188 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.366020 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.365995 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 17:44:29.366128 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.366073 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 17:44:29.366734 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.366709 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 17:44:29.366906 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.366888 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 17:44:29.366968 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.366946 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hgdk2\"" Jul 06 17:44:29.366968 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.366954 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d6575475f-sg844"] Jul 06 17:44:29.367143 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.367128 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.370116 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.370049 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.370229 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.370115 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 17:44:29.370229 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.370213 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 17:44:29.370342 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.370257 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 17:44:29.370402 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.370371 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:44:29.370592 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.370560 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dt54d\"" Jul 06 17:44:29.373158 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.373137 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 17:44:29.373700 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.373680 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4d8qp\"" Jul 06 17:44:29.373818 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.373721 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 17:44:29.373818 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.373769 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 17:44:29.373939 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.373850 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 17:44:29.374166 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.374139 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m"] Jul 06 17:44:29.374774 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.374753 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 17:44:29.375084 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.375065 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 17:44:29.384732 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.384707 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 17:44:29.386064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.386039 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-wg42h"] Jul 06 17:44:29.386961 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.386845 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 17:44:29.389090 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.389066 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d6575475f-sg844"] Jul 06 17:44:29.390556 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.390535 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cz4mf"] Jul 06 17:44:29.415578 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415549 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ef13f52-687a-47a1-afdf-a087c1b840f6-tmp\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.415578 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415583 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a73e9805-1648-4134-9d52-45c8640dcfb7-serving-cert\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.415724 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415605 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ab81ed4a-edf6-4033-a0ca-c4082f990911-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.415724 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415622 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a73e9805-1648-4134-9d52-45c8640dcfb7-config\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.415724 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415644 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a73e9805-1648-4134-9d52-45c8640dcfb7-trusted-ca\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.415724 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415671 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ef13f52-687a-47a1-afdf-a087c1b840f6-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.415724 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415714 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ef13f52-687a-47a1-afdf-a087c1b840f6-serving-cert\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.415940 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415755 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.415940 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415781 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42529\" (UniqueName: \"kubernetes.io/projected/ab81ed4a-edf6-4033-a0ca-c4082f990911-kube-api-access-42529\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.415940 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415808 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ef13f52-687a-47a1-afdf-a087c1b840f6-snapshots\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.415940 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415824 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl25v\" (UniqueName: \"kubernetes.io/projected/a73e9805-1648-4134-9d52-45c8640dcfb7-kube-api-access-gl25v\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.415940 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415891 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ef13f52-687a-47a1-afdf-a087c1b840f6-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.416142 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.415934 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lwrv\" (UniqueName: \"kubernetes.io/projected/7ef13f52-687a-47a1-afdf-a087c1b840f6-kube-api-access-8lwrv\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.453180 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.453156 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd"] Jul 06 17:44:29.455987 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.455972 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" Jul 06 17:44:29.457917 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.457889 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-959wx"] Jul 06 17:44:29.458030 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.458015 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rl4rq\"" Jul 06 17:44:29.458577 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.458560 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:44:29.458910 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.458893 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 17:44:29.460384 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.460371 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" Jul 06 17:44:29.462671 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.462654 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-96g6q\"" Jul 06 17:44:29.464340 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.464323 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd"] Jul 06 17:44:29.474374 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.474356 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-959wx"] Jul 06 17:44:29.516982 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.516962 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ef13f52-687a-47a1-afdf-a087c1b840f6-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.517064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.516987 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwrv\" (UniqueName: \"kubernetes.io/projected/7ef13f52-687a-47a1-afdf-a087c1b840f6-kube-api-access-8lwrv\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.517064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517007 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ef13f52-687a-47a1-afdf-a087c1b840f6-tmp\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.517064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517022 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a73e9805-1648-4134-9d52-45c8640dcfb7-serving-cert\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.517064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ab81ed4a-edf6-4033-a0ca-c4082f990911-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.517064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517057 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a73e9805-1648-4134-9d52-45c8640dcfb7-config\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.517255 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517081 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.517255 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517098 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-stats-auth\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.517255 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hntbr\" (UniqueName: \"kubernetes.io/projected/e717ae1c-49af-4b4c-83c7-362d14cab49b-kube-api-access-hntbr\") pod \"volume-data-source-validator-5d456fdbcd-b8nzd\" (UID: \"e717ae1c-49af-4b4c-83c7-362d14cab49b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" Jul 06 17:44:29.517255 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517133 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjg8x\" (UniqueName: \"kubernetes.io/projected/101351e0-a4a9-43a4-8297-d709b9e22d4c-kube-api-access-sjg8x\") pod \"network-check-source-989f8b8dc-959wx\" (UID: \"101351e0-a4a9-43a4-8297-d709b9e22d4c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" Jul 06 17:44:29.517255 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517158 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-default-certificate\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.517255 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a73e9805-1648-4134-9d52-45c8640dcfb7-trusted-ca\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517272 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ef13f52-687a-47a1-afdf-a087c1b840f6-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ef13f52-687a-47a1-afdf-a087c1b840f6-serving-cert\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517326 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517360 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517387 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjz88\" (UniqueName: \"kubernetes.io/projected/64a857a2-eb58-4f29-80cf-a2462430c591-kube-api-access-kjz88\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517423 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42529\" (UniqueName: \"kubernetes.io/projected/ab81ed4a-edf6-4033-a0ca-c4082f990911-kube-api-access-42529\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ef13f52-687a-47a1-afdf-a087c1b840f6-snapshots\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517514 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl25v\" (UniqueName: \"kubernetes.io/projected/a73e9805-1648-4134-9d52-45c8640dcfb7-kube-api-access-gl25v\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.517562 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:29.517521 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:29.518030 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:29.517572 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls podName:ab81ed4a-edf6-4033-a0ca-c4082f990911 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:30.01755504 +0000 UTC m=+237.423087954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fh97m" (UID: "ab81ed4a-edf6-4033-a0ca-c4082f990911") : secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:29.518030 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517657 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ef13f52-687a-47a1-afdf-a087c1b840f6-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.518030 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517423 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ef13f52-687a-47a1-afdf-a087c1b840f6-tmp\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.518030 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517833 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ab81ed4a-edf6-4033-a0ca-c4082f990911-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.518030 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.517988 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a73e9805-1648-4134-9d52-45c8640dcfb7-config\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.518295 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.518139 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a73e9805-1648-4134-9d52-45c8640dcfb7-trusted-ca\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.518295 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.518273 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ef13f52-687a-47a1-afdf-a087c1b840f6-snapshots\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.518580 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.518563 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ef13f52-687a-47a1-afdf-a087c1b840f6-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.519836 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.519800 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ef13f52-687a-47a1-afdf-a087c1b840f6-serving-cert\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.520125 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.520106 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a73e9805-1648-4134-9d52-45c8640dcfb7-serving-cert\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.523626 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.523610 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lwrv\" (UniqueName: \"kubernetes.io/projected/7ef13f52-687a-47a1-afdf-a087c1b840f6-kube-api-access-8lwrv\") pod \"insights-operator-5c847dcfc4-cz4mf\" (UID: \"7ef13f52-687a-47a1-afdf-a087c1b840f6\") " pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.530269 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.530241 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl25v\" (UniqueName: \"kubernetes.io/projected/a73e9805-1648-4134-9d52-45c8640dcfb7-kube-api-access-gl25v\") pod \"console-operator-68b757fb67-wg42h\" (UID: \"a73e9805-1648-4134-9d52-45c8640dcfb7\") " pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.530378 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.530362 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42529\" (UniqueName: \"kubernetes.io/projected/ab81ed4a-edf6-4033-a0ca-c4082f990911-kube-api-access-42529\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:29.618017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.617960 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.618017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.617992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-stats-auth\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.618161 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.618018 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hntbr\" (UniqueName: \"kubernetes.io/projected/e717ae1c-49af-4b4c-83c7-362d14cab49b-kube-api-access-hntbr\") pod \"volume-data-source-validator-5d456fdbcd-b8nzd\" (UID: \"e717ae1c-49af-4b4c-83c7-362d14cab49b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" Jul 06 17:44:29.618161 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.618042 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjg8x\" (UniqueName: \"kubernetes.io/projected/101351e0-a4a9-43a4-8297-d709b9e22d4c-kube-api-access-sjg8x\") pod \"network-check-source-989f8b8dc-959wx\" (UID: \"101351e0-a4a9-43a4-8297-d709b9e22d4c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" Jul 06 17:44:29.618161 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.618071 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-default-certificate\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.618161 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:29.618133 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:30.118115044 +0000 UTC m=+237.523647966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : configmap references non-existent config key: service-ca.crt Jul 06 17:44:29.618375 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.618159 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.618375 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.618209 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjz88\" (UniqueName: \"kubernetes.io/projected/64a857a2-eb58-4f29-80cf-a2462430c591-kube-api-access-kjz88\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.618375 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:29.618288 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 17:44:29.618375 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:29.618375 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:30.11835776 +0000 UTC m=+237.523890664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : secret "router-metrics-certs-default" not found Jul 06 17:44:29.619989 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.619971 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-stats-auth\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.620112 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.620096 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-default-certificate\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.626049 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.626024 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjg8x\" (UniqueName: \"kubernetes.io/projected/101351e0-a4a9-43a4-8297-d709b9e22d4c-kube-api-access-sjg8x\") pod \"network-check-source-989f8b8dc-959wx\" (UID: \"101351e0-a4a9-43a4-8297-d709b9e22d4c\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" Jul 06 17:44:29.626125 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.626033 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hntbr\" (UniqueName: \"kubernetes.io/projected/e717ae1c-49af-4b4c-83c7-362d14cab49b-kube-api-access-hntbr\") pod \"volume-data-source-validator-5d456fdbcd-b8nzd\" (UID: \"e717ae1c-49af-4b4c-83c7-362d14cab49b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" Jul 06 17:44:29.626400 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.626386 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjz88\" (UniqueName: \"kubernetes.io/projected/64a857a2-eb58-4f29-80cf-a2462430c591-kube-api-access-kjz88\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:29.683568 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.683549 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" Jul 06 17:44:29.691567 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.691550 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:29.765527 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.765500 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" Jul 06 17:44:29.770845 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.770694 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" Jul 06 17:44:29.825959 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.825836 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cz4mf"] Jul 06 17:44:29.829644 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:44:29.829598 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef13f52_687a_47a1_afdf_a087c1b840f6.slice/crio-85b9024c15ca78cddc7c8314f06392bf5b42c2e14ca682fc12a1cbe8b6966ea1 WatchSource:0}: Error finding container 85b9024c15ca78cddc7c8314f06392bf5b42c2e14ca682fc12a1cbe8b6966ea1: Status 404 returned error can't find the container with id 85b9024c15ca78cddc7c8314f06392bf5b42c2e14ca682fc12a1cbe8b6966ea1 Jul 06 17:44:29.847448 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.847381 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-wg42h"] Jul 06 17:44:29.850570 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:44:29.850543 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73e9805_1648_4134_9d52_45c8640dcfb7.slice/crio-d50d84c41bd8e322f8630420bcc8561cf6559cbde77691e08097ab278b608a58 WatchSource:0}: Error finding container d50d84c41bd8e322f8630420bcc8561cf6559cbde77691e08097ab278b608a58: Status 404 returned error can't find the container with id d50d84c41bd8e322f8630420bcc8561cf6559cbde77691e08097ab278b608a58 Jul 06 17:44:29.878111 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.878065 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" event={"ID":"a73e9805-1648-4134-9d52-45c8640dcfb7","Type":"ContainerStarted","Data":"d50d84c41bd8e322f8630420bcc8561cf6559cbde77691e08097ab278b608a58"} Jul 06 17:44:29.879449 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.879380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" event={"ID":"7ef13f52-687a-47a1-afdf-a087c1b840f6","Type":"ContainerStarted","Data":"85b9024c15ca78cddc7c8314f06392bf5b42c2e14ca682fc12a1cbe8b6966ea1"} Jul 06 17:44:29.899025 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.899001 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-959wx"] Jul 06 17:44:29.901581 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:44:29.901555 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101351e0_a4a9_43a4_8297_d709b9e22d4c.slice/crio-40206ba1f761f6b234a25def4e8a0acfbeb2fa282fd02f2f3306e2dcbbfa5009 WatchSource:0}: Error finding container 40206ba1f761f6b234a25def4e8a0acfbeb2fa282fd02f2f3306e2dcbbfa5009: Status 404 returned error can't find the container with id 40206ba1f761f6b234a25def4e8a0acfbeb2fa282fd02f2f3306e2dcbbfa5009 Jul 06 17:44:29.915360 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:29.915342 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd"] Jul 06 17:44:29.918899 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:44:29.918877 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode717ae1c_49af_4b4c_83c7_362d14cab49b.slice/crio-877903a5917c64b9e6dda880392b0d609eab0282c47a5c9956dd42a01461bbec WatchSource:0}: Error finding container 877903a5917c64b9e6dda880392b0d609eab0282c47a5c9956dd42a01461bbec: Status 404 returned error can't find the container with id 877903a5917c64b9e6dda880392b0d609eab0282c47a5c9956dd42a01461bbec Jul 06 17:44:30.020009 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:30.019984 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:30.020169 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:30.020149 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:30.020232 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:30.020222 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls podName:ab81ed4a-edf6-4033-a0ca-c4082f990911 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:31.020200903 +0000 UTC m=+238.425733812 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fh97m" (UID: "ab81ed4a-edf6-4033-a0ca-c4082f990911") : secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:30.121027 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:30.120977 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:30.121106 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:30.121042 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:30.121140 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:30.121111 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 17:44:30.121181 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:30.121135 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:31.121121876 +0000 UTC m=+238.526654780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : configmap references non-existent config key: service-ca.crt Jul 06 17:44:30.121181 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:30.121153 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:31.1211477 +0000 UTC m=+238.526680604 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : secret "router-metrics-certs-default" not found Jul 06 17:44:30.884101 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:30.884065 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" event={"ID":"e717ae1c-49af-4b4c-83c7-362d14cab49b","Type":"ContainerStarted","Data":"877903a5917c64b9e6dda880392b0d609eab0282c47a5c9956dd42a01461bbec"} Jul 06 17:44:30.885737 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:30.885695 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" event={"ID":"101351e0-a4a9-43a4-8297-d709b9e22d4c","Type":"ContainerStarted","Data":"da5fa454aa52d20eff44ab9b155a4eed3b1e21b44d0fcfd88012aa11471f3e47"} Jul 06 17:44:30.885871 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:30.885749 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" event={"ID":"101351e0-a4a9-43a4-8297-d709b9e22d4c","Type":"ContainerStarted","Data":"40206ba1f761f6b234a25def4e8a0acfbeb2fa282fd02f2f3306e2dcbbfa5009"} Jul 06 17:44:30.900249 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:30.900204 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-959wx" podStartSLOduration=1.900188769 podStartE2EDuration="1.900188769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:44:30.899642384 +0000 UTC m=+238.305175311" watchObservedRunningTime="2026-07-06 17:44:30.900188769 +0000 UTC m=+238.305721673" Jul 06 17:44:31.030351 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:31.030307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:31.030704 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:31.030684 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:31.030808 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:31.030757 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls podName:ab81ed4a-edf6-4033-a0ca-c4082f990911 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:33.030737756 +0000 UTC m=+240.436270681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fh97m" (UID: "ab81ed4a-edf6-4033-a0ca-c4082f990911") : secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:31.132022 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:31.131809 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:31.132022 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:31.131895 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:31.132022 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:31.132007 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 17:44:31.132022 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:31.132021 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:33.131997716 +0000 UTC m=+240.537530636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : configmap references non-existent config key: service-ca.crt Jul 06 17:44:31.132372 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:31.132058 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:33.132041429 +0000 UTC m=+240.537574337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : secret "router-metrics-certs-default" not found Jul 06 17:44:31.890923 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:31.890852 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" event={"ID":"e717ae1c-49af-4b4c-83c7-362d14cab49b","Type":"ContainerStarted","Data":"46d65aa62802d03e2003c7c5eefd25fddc3ad20e930bb8e020adf23fa0d84a05"} Jul 06 17:44:31.905191 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:31.905148 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-b8nzd" podStartSLOduration=1.558820417 podStartE2EDuration="2.90513418s" totalImagesPullingTime="1.346313763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:44:31.904418534 +0000 UTC m=+239.309951459" watchObservedRunningTime="2026-07-06 17:44:31.90513418 +0000 UTC m=+239.310667163" Jul 06 17:44:32.895930 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:32.895879 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/0.log" Jul 06 17:44:32.895930 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:32.895916 2558 generic.go:356] "Generic (PLEG): container finished" podID="a73e9805-1648-4134-9d52-45c8640dcfb7" containerID="fff3ee76dbc5ecff9e9f72c1b6c8c894bc63eee7aea5b2df7e0bf38d035ccec2" exitCode=255 Jul 06 17:44:32.896303 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:32.895972 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" event={"ID":"a73e9805-1648-4134-9d52-45c8640dcfb7","Type":"ContainerDied","Data":"fff3ee76dbc5ecff9e9f72c1b6c8c894bc63eee7aea5b2df7e0bf38d035ccec2"} Jul 06 17:44:32.896303 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:32.896207 2558 scope.go:117] "RemoveContainer" containerID="fff3ee76dbc5ecff9e9f72c1b6c8c894bc63eee7aea5b2df7e0bf38d035ccec2" Jul 06 17:44:32.897493 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:32.897473 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" event={"ID":"7ef13f52-687a-47a1-afdf-a087c1b840f6","Type":"ContainerStarted","Data":"49fcca0dce78fb387572ae357b11cc346c79cfe98bc1b18c713d2c8908c4269d"} Jul 06 17:44:32.923320 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:32.923283 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" podStartSLOduration=1.137380744 podStartE2EDuration="3.923270687s" totalImagesPullingTime="2.785889943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:44:32.923007425 +0000 UTC m=+240.328540348" watchObservedRunningTime="2026-07-06 17:44:32.923270687 +0000 UTC m=+240.328803612" Jul 06 17:44:33.053227 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.053195 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:33.053347 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.053315 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:33.053411 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.053379 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls podName:ab81ed4a-edf6-4033-a0ca-c4082f990911 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:37.05335834 +0000 UTC m=+244.458891247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fh97m" (UID: "ab81ed4a-edf6-4033-a0ca-c4082f990911") : secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:33.154303 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.154249 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:33.154303 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.154299 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:33.154424 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.154381 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 17:44:33.154424 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.154406 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:37.154387708 +0000 UTC m=+244.559920617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : configmap references non-existent config key: service-ca.crt Jul 06 17:44:33.154514 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.154452 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:37.154423085 +0000 UTC m=+244.559955994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : secret "router-metrics-certs-default" not found Jul 06 17:44:33.536145 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.536113 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hcrz9"] Jul 06 17:44:33.539749 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.539731 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.543297 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.543277 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 17:44:33.543395 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.543280 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 17:44:33.543395 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.543280 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xts8l\"" Jul 06 17:44:33.555486 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.555469 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hcrz9"] Jul 06 17:44:33.658100 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.658078 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-crio-socket\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.658192 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.658122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj7gh\" (UniqueName: \"kubernetes.io/projected/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-api-access-jj7gh\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.658192 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.658149 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-data-volume\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.658287 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.658263 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.658338 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.658319 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759289 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.759265 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-crio-socket\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759376 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.759294 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj7gh\" (UniqueName: \"kubernetes.io/projected/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-api-access-jj7gh\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759376 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.759313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-data-volume\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759475 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.759394 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-crio-socket\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759475 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.759402 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759553 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.759498 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:44:33.759553 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.759503 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759553 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.759546 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:34.259531166 +0000 UTC m=+241.665064070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : secret "insights-runtime-extractor-tls" not found Jul 06 17:44:33.759681 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.759599 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-data-volume\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.759681 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.759605 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:44:33.759681 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.759650 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:34.259638108 +0000 UTC m=+241.665171013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : configmap "kube-rbac-proxy" not found Jul 06 17:44:33.766884 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.766865 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj7gh\" (UniqueName: \"kubernetes.io/projected/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-api-access-jj7gh\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:33.902227 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.902175 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:44:33.902618 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.902602 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/0.log" Jul 06 17:44:33.902679 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.902646 2558 generic.go:356] "Generic (PLEG): container finished" podID="a73e9805-1648-4134-9d52-45c8640dcfb7" containerID="f645d6b22d3746c5d469717346050dfa0c851068288542d3ab228ecdb1afc107" exitCode=255 Jul 06 17:44:33.902755 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.902731 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" event={"ID":"a73e9805-1648-4134-9d52-45c8640dcfb7","Type":"ContainerDied","Data":"f645d6b22d3746c5d469717346050dfa0c851068288542d3ab228ecdb1afc107"} Jul 06 17:44:33.902804 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.902789 2558 scope.go:117] "RemoveContainer" containerID="fff3ee76dbc5ecff9e9f72c1b6c8c894bc63eee7aea5b2df7e0bf38d035ccec2" Jul 06 17:44:33.902905 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:33.902890 2558 scope.go:117] "RemoveContainer" containerID="f645d6b22d3746c5d469717346050dfa0c851068288542d3ab228ecdb1afc107" Jul 06 17:44:33.903054 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:33.903035 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-wg42h_openshift-console-operator(a73e9805-1648-4134-9d52-45c8640dcfb7)\"" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" podUID="a73e9805-1648-4134-9d52-45c8640dcfb7" Jul 06 17:44:34.263720 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:34.263696 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:34.263842 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:34.263743 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:34.263842 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:34.263826 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:44:34.263924 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:34.263843 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:44:34.263924 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:34.263887 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:35.263872449 +0000 UTC m=+242.669405357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : secret "insights-runtime-extractor-tls" not found Jul 06 17:44:34.263924 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:34.263902 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:35.263895341 +0000 UTC m=+242.669428245 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : configmap "kube-rbac-proxy" not found Jul 06 17:44:34.907072 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:34.907043 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:44:34.907489 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:34.907332 2558 scope.go:117] "RemoveContainer" containerID="f645d6b22d3746c5d469717346050dfa0c851068288542d3ab228ecdb1afc107" Jul 06 17:44:34.907533 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:34.907484 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-wg42h_openshift-console-operator(a73e9805-1648-4134-9d52-45c8640dcfb7)\"" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" podUID="a73e9805-1648-4134-9d52-45c8640dcfb7" Jul 06 17:44:35.273655 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:35.273625 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:35.273803 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:35.273676 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:35.273803 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:35.273767 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:44:35.273803 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:35.273770 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:44:35.273896 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:35.273829 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:37.273815727 +0000 UTC m=+244.679348632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : configmap "kube-rbac-proxy" not found Jul 06 17:44:35.273896 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:35.273841 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:37.273835892 +0000 UTC m=+244.679368796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : secret "insights-runtime-extractor-tls" not found Jul 06 17:44:35.751375 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:35.751351 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vx2ng_a25767e7-609c-42a7-8b8b-f793ee69d50f/dns-node-resolver/0.log" Jul 06 17:44:36.752052 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:36.752025 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-th6lw_014ceecb-1938-4660-b2fa-7c17bb94b782/node-ca/0.log" Jul 06 17:44:37.086743 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:37.086668 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:37.086881 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.086815 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:37.086932 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.086889 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls podName:ab81ed4a-edf6-4033-a0ca-c4082f990911 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:45.086873745 +0000 UTC m=+252.492406649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fh97m" (UID: "ab81ed4a-edf6-4033-a0ca-c4082f990911") : secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:37.151350 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:37.151315 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jpln9_3a2ef3dd-ede8-498e-ae66-0941952e042a/serve-healthcheck-canary/0.log" Jul 06 17:44:37.187676 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:37.187647 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:37.187793 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:37.187693 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:37.187856 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.187819 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:45.187802973 +0000 UTC m=+252.593335876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : configmap references non-existent config key: service-ca.crt Jul 06 17:44:37.187928 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.187906 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 17:44:37.187980 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.187964 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:45.187951642 +0000 UTC m=+252.593484546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : secret "router-metrics-certs-default" not found Jul 06 17:44:37.288467 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:37.288437 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:37.288592 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:37.288495 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:37.288592 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.288575 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:44:37.288688 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.288595 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:44:37.288688 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.288635 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:41.288620423 +0000 UTC m=+248.694153327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : secret "insights-runtime-extractor-tls" not found Jul 06 17:44:37.288688 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:37.288652 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:41.288644546 +0000 UTC m=+248.694177450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : configmap "kube-rbac-proxy" not found Jul 06 17:44:39.691700 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:39.691620 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:39.691700 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:39.691679 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:39.692081 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:39.691986 2558 scope.go:117] "RemoveContainer" containerID="f645d6b22d3746c5d469717346050dfa0c851068288542d3ab228ecdb1afc107" Jul 06 17:44:39.692157 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:39.692132 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-wg42h_openshift-console-operator(a73e9805-1648-4134-9d52-45c8640dcfb7)\"" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" podUID="a73e9805-1648-4134-9d52-45c8640dcfb7" Jul 06 17:44:41.319771 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:41.319737 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:41.320201 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:41.319784 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:41.320201 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:41.319888 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 17:44:41.320201 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:41.319958 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:49.319944714 +0000 UTC m=+256.725477618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : configmap "kube-rbac-proxy" not found Jul 06 17:44:41.320201 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:41.319896 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:44:41.320201 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:41.320055 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:49.320034908 +0000 UTC m=+256.725567819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : secret "insights-runtime-extractor-tls" not found Jul 06 17:44:43.838008 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:43.837977 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:44:43.838395 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:43.838119 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 17:44:43.838395 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:43.838195 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs podName:e7646b7b-d22f-4fab-9609-1c3beb2632e1 nodeName:}" failed. No retries permitted until 2026-07-06 17:46:45.838178938 +0000 UTC m=+373.243711841 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs") pod "network-metrics-daemon-b9gd9" (UID: "e7646b7b-d22f-4fab-9609-1c3beb2632e1") : secret "metrics-daemon-secret" not found Jul 06 17:44:45.153415 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:45.149045 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:44:45.153415 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:45.149661 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:45.153415 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:45.149748 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls podName:ab81ed4a-edf6-4033-a0ca-c4082f990911 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:01.149725489 +0000 UTC m=+268.555258413 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fh97m" (UID: "ab81ed4a-edf6-4033-a0ca-c4082f990911") : secret "cluster-monitoring-operator-tls" not found Jul 06 17:44:45.250286 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:45.250256 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:45.250475 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:45.250307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:45.250538 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:45.250512 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle podName:64a857a2-eb58-4f29-80cf-a2462430c591 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:01.250491165 +0000 UTC m=+268.656024086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle") pod "router-default-7d6575475f-sg844" (UID: "64a857a2-eb58-4f29-80cf-a2462430c591") : configmap references non-existent config key: service-ca.crt Jul 06 17:44:45.252794 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:45.252765 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a857a2-eb58-4f29-80cf-a2462430c591-metrics-certs\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:44:49.382787 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:49.382745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:49.383152 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:49.382800 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:49.383152 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:49.382889 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 17:44:49.383152 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:44:49.382950 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls podName:9e6ad0b1-1fdd-495d-9fe7-225f1112d537 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:05.382936943 +0000 UTC m=+272.788469847 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcrz9" (UID: "9e6ad0b1-1fdd-495d-9fe7-225f1112d537") : secret "insights-runtime-extractor-tls" not found Jul 06 17:44:49.383283 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:49.383259 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:44:54.064653 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:54.064623 2558 scope.go:117] "RemoveContainer" containerID="f645d6b22d3746c5d469717346050dfa0c851068288542d3ab228ecdb1afc107" Jul 06 17:44:54.966774 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:54.966695 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:44:54.966774 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:54.966761 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" event={"ID":"a73e9805-1648-4134-9d52-45c8640dcfb7","Type":"ContainerStarted","Data":"34b6dfb8478f10b2939a5eeaf3156d7faa694d3664aa4b571dfdcb689957f7ea"} Jul 06 17:44:54.967081 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:54.967054 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:54.971615 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:54.971594 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" Jul 06 17:44:54.987803 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:54.987767 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-wg42h" podStartSLOduration=23.220271392 podStartE2EDuration="25.987754272s" totalImagesPullingTime="2.76748288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:44:54.987154098 +0000 UTC m=+262.392687018" watchObservedRunningTime="2026-07-06 17:44:54.987754272 +0000 UTC m=+262.393287198" Jul 06 17:44:55.158222 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.158202 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-jqrq5"] Jul 06 17:44:55.161024 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.161009 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-jqrq5" Jul 06 17:44:55.163228 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.163209 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 17:44:55.163336 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.163272 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 17:44:55.163581 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.163564 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-spt2h\"" Jul 06 17:44:55.175600 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.175577 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-jqrq5"] Jul 06 17:44:55.203566 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.203545 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66c479597f-bqxb2"] Jul 06 17:44:55.206343 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.206329 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.207990 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.207973 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 17:44:55.208243 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.208222 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 17:44:55.208546 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.208531 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v256q\"" Jul 06 17:44:55.208615 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.208572 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 17:44:55.213188 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.213172 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 17:44:55.215655 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.215638 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66c479597f-bqxb2"] Jul 06 17:44:55.224663 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224612 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-registry-tls\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.224663 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224640 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8dzh\" (UniqueName: \"kubernetes.io/projected/b7e9b308-ae61-40eb-83e9-1a54661dc34c-kube-api-access-g8dzh\") pod \"downloads-685bfb5f64-jqrq5\" (UID: \"b7e9b308-ae61-40eb-83e9-1a54661dc34c\") " pod="openshift-console/downloads-685bfb5f64-jqrq5" Jul 06 17:44:55.224766 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224667 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n42wq\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-kube-api-access-n42wq\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.224805 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224785 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-bound-sa-token\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.224839 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224820 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fd82183-ab7a-4e74-97e0-447e89300893-trusted-ca\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.224869 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224839 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fd82183-ab7a-4e74-97e0-447e89300893-installation-pull-secrets\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.224869 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224857 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fd82183-ab7a-4e74-97e0-447e89300893-image-registry-private-configuration\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.224937 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224881 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fd82183-ab7a-4e74-97e0-447e89300893-ca-trust-extracted\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.224937 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.224929 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fd82183-ab7a-4e74-97e0-447e89300893-registry-certificates\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.325719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.325695 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fd82183-ab7a-4e74-97e0-447e89300893-trusted-ca\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.325817 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.325730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fd82183-ab7a-4e74-97e0-447e89300893-installation-pull-secrets\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.325817 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.325756 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fd82183-ab7a-4e74-97e0-447e89300893-image-registry-private-configuration\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.325817 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.325792 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fd82183-ab7a-4e74-97e0-447e89300893-ca-trust-extracted\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.325968 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.325858 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fd82183-ab7a-4e74-97e0-447e89300893-registry-certificates\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.326017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.325971 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-registry-tls\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.326017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.326001 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzh\" (UniqueName: \"kubernetes.io/projected/b7e9b308-ae61-40eb-83e9-1a54661dc34c-kube-api-access-g8dzh\") pod \"downloads-685bfb5f64-jqrq5\" (UID: \"b7e9b308-ae61-40eb-83e9-1a54661dc34c\") " pod="openshift-console/downloads-685bfb5f64-jqrq5" Jul 06 17:44:55.326122 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.326049 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n42wq\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-kube-api-access-n42wq\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.326179 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.326160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-bound-sa-token\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.326286 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.326267 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1fd82183-ab7a-4e74-97e0-447e89300893-ca-trust-extracted\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.326682 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.326660 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1fd82183-ab7a-4e74-97e0-447e89300893-registry-certificates\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.327532 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.327510 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1fd82183-ab7a-4e74-97e0-447e89300893-trusted-ca\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.328116 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.328094 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1fd82183-ab7a-4e74-97e0-447e89300893-image-registry-private-configuration\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.328252 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.328234 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1fd82183-ab7a-4e74-97e0-447e89300893-installation-pull-secrets\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.328536 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.328519 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-registry-tls\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.340784 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.340752 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-bound-sa-token\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.341175 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.341153 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8dzh\" (UniqueName: \"kubernetes.io/projected/b7e9b308-ae61-40eb-83e9-1a54661dc34c-kube-api-access-g8dzh\") pod \"downloads-685bfb5f64-jqrq5\" (UID: \"b7e9b308-ae61-40eb-83e9-1a54661dc34c\") " pod="openshift-console/downloads-685bfb5f64-jqrq5" Jul 06 17:44:55.341226 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.341213 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n42wq\" (UniqueName: \"kubernetes.io/projected/1fd82183-ab7a-4e74-97e0-447e89300893-kube-api-access-n42wq\") pod \"image-registry-66c479597f-bqxb2\" (UID: \"1fd82183-ab7a-4e74-97e0-447e89300893\") " pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.469909 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.469886 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-jqrq5" Jul 06 17:44:55.515782 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.515759 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.610573 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.610546 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-jqrq5"] Jul 06 17:44:55.610701 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:44:55.610667 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e9b308_ae61_40eb_83e9_1a54661dc34c.slice/crio-388fc070f0095d7a0a205f918d11dfd5a76df8db9aa979d961a04b3bedf4612a WatchSource:0}: Error finding container 388fc070f0095d7a0a205f918d11dfd5a76df8db9aa979d961a04b3bedf4612a: Status 404 returned error can't find the container with id 388fc070f0095d7a0a205f918d11dfd5a76df8db9aa979d961a04b3bedf4612a Jul 06 17:44:55.659333 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.659303 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66c479597f-bqxb2"] Jul 06 17:44:55.662879 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:44:55.662858 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd82183_ab7a_4e74_97e0_447e89300893.slice/crio-aa4a934159b27aef573d8bcc89206ea26c0756b1773e4044b9a83394d674c555 WatchSource:0}: Error finding container aa4a934159b27aef573d8bcc89206ea26c0756b1773e4044b9a83394d674c555: Status 404 returned error can't find the container with id aa4a934159b27aef573d8bcc89206ea26c0756b1773e4044b9a83394d674c555 Jul 06 17:44:55.972017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.971978 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" event={"ID":"1fd82183-ab7a-4e74-97e0-447e89300893","Type":"ContainerStarted","Data":"19dfec4a183932b1d5297044d1cd307de338180dc1c752b076c712052c7d019b"} Jul 06 17:44:55.972224 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.972078 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" event={"ID":"1fd82183-ab7a-4e74-97e0-447e89300893","Type":"ContainerStarted","Data":"aa4a934159b27aef573d8bcc89206ea26c0756b1773e4044b9a83394d674c555"} Jul 06 17:44:55.972224 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.972109 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:44:55.973172 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.973145 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-jqrq5" event={"ID":"b7e9b308-ae61-40eb-83e9-1a54661dc34c","Type":"ContainerStarted","Data":"388fc070f0095d7a0a205f918d11dfd5a76df8db9aa979d961a04b3bedf4612a"} Jul 06 17:44:55.990818 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:44:55.990773 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podStartSLOduration=0.990760315 podStartE2EDuration="990.760315ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:44:55.989568615 +0000 UTC m=+263.395101566" watchObservedRunningTime="2026-07-06 17:44:55.990760315 +0000 UTC m=+263.396293253" Jul 06 17:45:01.172361 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.172326 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:45:01.174727 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.174705 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ab81ed4a-edf6-4033-a0ca-c4082f990911-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fh97m\" (UID: \"ab81ed4a-edf6-4033-a0ca-c4082f990911\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:45:01.273040 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.273006 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:45:01.273585 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.273566 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64a857a2-eb58-4f29-80cf-a2462430c591-service-ca-bundle\") pod \"router-default-7d6575475f-sg844\" (UID: \"64a857a2-eb58-4f29-80cf-a2462430c591\") " pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:45:01.474692 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.474626 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ttxk6\"" Jul 06 17:45:01.483252 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.483232 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" Jul 06 17:45:01.498049 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.498024 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4d8qp\"" Jul 06 17:45:01.506577 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.506555 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:45:01.631025 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.630998 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m"] Jul 06 17:45:01.633482 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:01.633446 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab81ed4a_edf6_4033_a0ca_c4082f990911.slice/crio-89be889a1d3586df8d34e5000cd484d710ea1c1796b7c78392f9a364c3c1fde8 WatchSource:0}: Error finding container 89be889a1d3586df8d34e5000cd484d710ea1c1796b7c78392f9a364c3c1fde8: Status 404 returned error can't find the container with id 89be889a1d3586df8d34e5000cd484d710ea1c1796b7c78392f9a364c3c1fde8 Jul 06 17:45:01.644781 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.644756 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d6575475f-sg844"] Jul 06 17:45:01.647017 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:01.646990 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a857a2_eb58_4f29_80cf_a2462430c591.slice/crio-f469570900e38b27229ef3008f48546258ac8568d4918ab48651b21c46fd2754 WatchSource:0}: Error finding container f469570900e38b27229ef3008f48546258ac8568d4918ab48651b21c46fd2754: Status 404 returned error can't find the container with id f469570900e38b27229ef3008f48546258ac8568d4918ab48651b21c46fd2754 Jul 06 17:45:01.996567 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.996532 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d6575475f-sg844" event={"ID":"64a857a2-eb58-4f29-80cf-a2462430c591","Type":"ContainerStarted","Data":"25a16a7b78461cf193f57d7a1bbe383a312a8a46fe541f184451a8ce7f34a222"} Jul 06 17:45:01.996752 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.996592 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d6575475f-sg844" event={"ID":"64a857a2-eb58-4f29-80cf-a2462430c591","Type":"ContainerStarted","Data":"f469570900e38b27229ef3008f48546258ac8568d4918ab48651b21c46fd2754"} Jul 06 17:45:01.997825 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:01.997799 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" event={"ID":"ab81ed4a-edf6-4033-a0ca-c4082f990911","Type":"ContainerStarted","Data":"89be889a1d3586df8d34e5000cd484d710ea1c1796b7c78392f9a364c3c1fde8"} Jul 06 17:45:02.015418 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.015373 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7d6575475f-sg844" podStartSLOduration=33.015357351 podStartE2EDuration="33.015357351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:02.0138661 +0000 UTC m=+269.419399051" watchObservedRunningTime="2026-07-06 17:45:02.015357351 +0000 UTC m=+269.420890280" Jul 06 17:45:02.416567 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.416505 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b96f44445-ccbxj"] Jul 06 17:45:02.419684 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.419665 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.421825 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.421808 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 17:45:02.421920 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.421868 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 17:45:02.421920 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.421868 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kbbzg\"" Jul 06 17:45:02.422065 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.422050 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 17:45:02.422151 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.422135 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 17:45:02.422345 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.422329 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 17:45:02.429119 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.429097 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b96f44445-ccbxj"] Jul 06 17:45:02.481495 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.481468 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-config\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.481595 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.481526 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-serving-cert\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.481636 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.481587 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-oauth-config\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.481636 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.481616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvgcj\" (UniqueName: \"kubernetes.io/projected/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-kube-api-access-vvgcj\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.481636 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.481633 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-service-ca\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.481745 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.481654 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-oauth-serving-cert\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.507631 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.507608 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:45:02.509940 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.509921 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:45:02.582097 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.582070 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-config\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.582206 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.582117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-serving-cert\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.582314 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.582295 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-oauth-config\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.582367 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.582333 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvgcj\" (UniqueName: \"kubernetes.io/projected/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-kube-api-access-vvgcj\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.582367 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.582355 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-service-ca\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.582489 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.582388 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-oauth-serving-cert\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.582883 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.582855 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-config\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.583039 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.583015 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-oauth-serving-cert\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.583246 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.583226 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-service-ca\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.584624 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.584603 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-oauth-config\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.584872 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.584856 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-serving-cert\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.590885 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.590862 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvgcj\" (UniqueName: \"kubernetes.io/projected/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-kube-api-access-vvgcj\") pod \"console-7b96f44445-ccbxj\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.728665 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.728605 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:02.874621 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:02.874593 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b96f44445-ccbxj"] Jul 06 17:45:02.879350 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:02.879320 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06ff792_0efd_4f30_a7c2_c1fdffc5ed3d.slice/crio-fca93f193b0205eab72df0acb3927f753cdf9357befb3b224abe857ecf1ef2b1 WatchSource:0}: Error finding container fca93f193b0205eab72df0acb3927f753cdf9357befb3b224abe857ecf1ef2b1: Status 404 returned error can't find the container with id fca93f193b0205eab72df0acb3927f753cdf9357befb3b224abe857ecf1ef2b1 Jul 06 17:45:03.003670 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:03.003613 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b96f44445-ccbxj" event={"ID":"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d","Type":"ContainerStarted","Data":"fca93f193b0205eab72df0acb3927f753cdf9357befb3b224abe857ecf1ef2b1"} Jul 06 17:45:03.004012 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:03.003987 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:45:03.005499 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:03.005479 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d6575475f-sg844" Jul 06 17:45:05.013193 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:05.013104 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" event={"ID":"ab81ed4a-edf6-4033-a0ca-c4082f990911","Type":"ContainerStarted","Data":"a86d7df3bcc815cc1ad62fd87bb18e9548498d629e29f0f243aa8253f569a25a"} Jul 06 17:45:05.030094 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:05.030043 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fh97m" podStartSLOduration=33.605665223 podStartE2EDuration="36.030024672s" totalImagesPullingTime="2.424359449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:05.029067578 +0000 UTC m=+272.434600533" watchObservedRunningTime="2026-07-06 17:45:05.030024672 +0000 UTC m=+272.435557601" Jul 06 17:45:05.423934 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:05.423851 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:45:05.426360 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:05.426330 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e6ad0b1-1fdd-495d-9fe7-225f1112d537-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcrz9\" (UID: \"9e6ad0b1-1fdd-495d-9fe7-225f1112d537\") " pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:45:05.648736 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:05.648704 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hcrz9" Jul 06 17:45:06.883382 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:06.883355 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hcrz9"] Jul 06 17:45:06.886736 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:06.886711 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e6ad0b1_1fdd_495d_9fe7_225f1112d537.slice/crio-ca3c9081b5d2c843e862dc8eeae324685d90f8a786418057e795b8095f7c7aae WatchSource:0}: Error finding container ca3c9081b5d2c843e862dc8eeae324685d90f8a786418057e795b8095f7c7aae: Status 404 returned error can't find the container with id ca3c9081b5d2c843e862dc8eeae324685d90f8a786418057e795b8095f7c7aae Jul 06 17:45:07.023146 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:07.023042 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcrz9" event={"ID":"9e6ad0b1-1fdd-495d-9fe7-225f1112d537","Type":"ContainerStarted","Data":"ca3c9081b5d2c843e862dc8eeae324685d90f8a786418057e795b8095f7c7aae"} Jul 06 17:45:07.024799 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:07.024775 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b96f44445-ccbxj" event={"ID":"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d","Type":"ContainerStarted","Data":"b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba"} Jul 06 17:45:07.040489 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:07.040442 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b96f44445-ccbxj" podStartSLOduration=1.126748895 podStartE2EDuration="5.040412616s" totalImagesPullingTime="3.913663721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:07.039589625 +0000 UTC m=+274.445122552" watchObservedRunningTime="2026-07-06 17:45:07.040412616 +0000 UTC m=+274.445945540" Jul 06 17:45:08.629288 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.629255 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw"] Jul 06 17:45:08.639341 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.639313 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.640044 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.640018 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw"] Jul 06 17:45:08.641444 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.641409 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 17:45:08.641542 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.641415 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 17:45:08.642098 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.642078 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rcwmq\"" Jul 06 17:45:08.642218 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.642143 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 17:45:08.754749 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.754725 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6czkq\" (UniqueName: \"kubernetes.io/projected/d73f6985-e01f-4778-96b9-75a27f2cac4d-kube-api-access-6czkq\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.754876 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.754796 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d73f6985-e01f-4778-96b9-75a27f2cac4d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.754938 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.754867 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.754938 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.754915 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.855528 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.855498 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d73f6985-e01f-4778-96b9-75a27f2cac4d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.855670 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.855556 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.855670 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.855591 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.855670 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.855612 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6czkq\" (UniqueName: \"kubernetes.io/projected/d73f6985-e01f-4778-96b9-75a27f2cac4d-kube-api-access-6czkq\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.855828 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:45:08.855718 2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 17:45:08.855828 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:45:08.855791 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-tls podName:d73f6985-e01f-4778-96b9-75a27f2cac4d nodeName:}" failed. No retries permitted until 2026-07-06 17:45:09.35577192 +0000 UTC m=+276.761304826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-r2lnw" (UID: "d73f6985-e01f-4778-96b9-75a27f2cac4d") : secret "prometheus-operator-tls" not found Jul 06 17:45:08.856254 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.856234 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d73f6985-e01f-4778-96b9-75a27f2cac4d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.858017 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.857994 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:08.863506 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:08.863478 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6czkq\" (UniqueName: \"kubernetes.io/projected/d73f6985-e01f-4778-96b9-75a27f2cac4d-kube-api-access-6czkq\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:09.358956 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:09.358913 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:09.361691 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:09.361669 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d73f6985-e01f-4778-96b9-75a27f2cac4d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-r2lnw\" (UID: \"d73f6985-e01f-4778-96b9-75a27f2cac4d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:09.551093 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:09.551062 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" Jul 06 17:45:12.729439 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:12.729400 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:12.729881 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:12.729506 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:12.730939 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:12.730908 2558 patch_prober.go:28] interesting pod/console-7b96f44445-ccbxj container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" start-of-body= Jul 06 17:45:12.731067 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:12.730982 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b96f44445-ccbxj" podUID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" containerName="console" probeResult="failure" output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" Jul 06 17:45:15.521182 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:15.521151 2558 patch_prober.go:28] interesting pod/image-registry-66c479597f-bqxb2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 17:45:15.521643 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:15.521219 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podUID="1fd82183-ab7a-4e74-97e0-447e89300893" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 17:45:15.691505 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:45:15.691459 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mgxwp" podUID="a34e0be6-2c80-4817-80dd-5c4fbc90feb1" Jul 06 17:45:16.057579 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:16.057545 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mgxwp" Jul 06 17:45:16.981579 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:16.981550 2558 patch_prober.go:28] interesting pod/image-registry-66c479597f-bqxb2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 17:45:16.981996 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:16.981595 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podUID="1fd82183-ab7a-4e74-97e0-447e89300893" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 17:45:17.474135 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:17.474112 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw"] Jul 06 17:45:17.476253 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:17.476230 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73f6985_e01f_4778_96b9_75a27f2cac4d.slice/crio-1ee1c0961785ae5a10dc1e07a6ec932035a6bd01337d6936da04471111ff45c5 WatchSource:0}: Error finding container 1ee1c0961785ae5a10dc1e07a6ec932035a6bd01337d6936da04471111ff45c5: Status 404 returned error can't find the container with id 1ee1c0961785ae5a10dc1e07a6ec932035a6bd01337d6936da04471111ff45c5 Jul 06 17:45:18.068140 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.068101 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-jqrq5" event={"ID":"b7e9b308-ae61-40eb-83e9-1a54661dc34c","Type":"ContainerStarted","Data":"fe185d27df8e4a282b59e6b4748218d33ca4671c3fd7f0a1fa6074bef47ae25e"} Jul 06 17:45:18.069253 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.069104 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-jqrq5" Jul 06 17:45:18.071890 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.071826 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcrz9" event={"ID":"9e6ad0b1-1fdd-495d-9fe7-225f1112d537","Type":"ContainerStarted","Data":"8ad1d844d1edf0d092f9f7526fb1d3c04a2476142ef021c404b66abeee44f122"} Jul 06 17:45:18.073187 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.073156 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" event={"ID":"d73f6985-e01f-4778-96b9-75a27f2cac4d","Type":"ContainerStarted","Data":"1ee1c0961785ae5a10dc1e07a6ec932035a6bd01337d6936da04471111ff45c5"} Jul 06 17:45:18.081873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.081851 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-jqrq5" Jul 06 17:45:18.084904 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.084861 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-jqrq5" podStartSLOduration=1.312551082 podStartE2EDuration="23.084848393s" totalImagesPullingTime="21.772297311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:18.083843155 +0000 UTC m=+285.489376104" watchObservedRunningTime="2026-07-06 17:45:18.084848393 +0000 UTC m=+285.490381318" Jul 06 17:45:18.836704 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.836669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:45:18.840096 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:18.840068 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a34e0be6-2c80-4817-80dd-5c4fbc90feb1-metrics-tls\") pod \"dns-default-mgxwp\" (UID: \"a34e0be6-2c80-4817-80dd-5c4fbc90feb1\") " pod="openshift-dns/dns-default-mgxwp" Jul 06 17:45:19.060663 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:19.060629 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmt7g\"" Jul 06 17:45:19.068743 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:19.068718 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mgxwp" Jul 06 17:45:19.080931 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:19.080901 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcrz9" event={"ID":"9e6ad0b1-1fdd-495d-9fe7-225f1112d537","Type":"ContainerStarted","Data":"82770aee7483973293f223fbd27e46e63e6b31b86087806745110c0a40ea92b3"} Jul 06 17:45:19.519472 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:19.519337 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mgxwp"] Jul 06 17:45:19.525110 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:19.525065 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34e0be6_2c80_4817_80dd_5c4fbc90feb1.slice/crio-9350b858bf331b211174dc6d33aaea85a8bf3081b393aa8dd8cb146648aa0c5d WatchSource:0}: Error finding container 9350b858bf331b211174dc6d33aaea85a8bf3081b393aa8dd8cb146648aa0c5d: Status 404 returned error can't find the container with id 9350b858bf331b211174dc6d33aaea85a8bf3081b393aa8dd8cb146648aa0c5d Jul 06 17:45:20.087845 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:20.087724 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" event={"ID":"d73f6985-e01f-4778-96b9-75a27f2cac4d","Type":"ContainerStarted","Data":"d76b37ba65ece59340b94d02b64c683d26e66fdc6c7b170f7ee248a426eb3ce0"} Jul 06 17:45:20.087845 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:20.087790 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" event={"ID":"d73f6985-e01f-4778-96b9-75a27f2cac4d","Type":"ContainerStarted","Data":"9d4b9a22bd747c670b3bb377cf5977ac5571f9b80440f677d40e8c22189868bd"} Jul 06 17:45:20.089959 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:20.089875 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mgxwp" event={"ID":"a34e0be6-2c80-4817-80dd-5c4fbc90feb1","Type":"ContainerStarted","Data":"9350b858bf331b211174dc6d33aaea85a8bf3081b393aa8dd8cb146648aa0c5d"} Jul 06 17:45:21.114344 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:21.114277 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-r2lnw" podStartSLOduration=10.750291047 podStartE2EDuration="13.114255555s" totalImagesPullingTime="2.363964508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:45:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:21.11245321 +0000 UTC m=+288.517986152" watchObservedRunningTime="2026-07-06 17:45:21.114255555 +0000 UTC m=+288.519788482" Jul 06 17:45:22.729922 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:22.729874 2558 patch_prober.go:28] interesting pod/console-7b96f44445-ccbxj container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" start-of-body= Jul 06 17:45:22.730315 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:22.729961 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b96f44445-ccbxj" podUID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" containerName="console" probeResult="failure" output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" Jul 06 17:45:22.993513 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:22.993479 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8pg8s"] Jul 06 17:45:23.009267 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.009236 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.011670 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.011647 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 17:45:23.011787 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.011705 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 17:45:23.011787 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.011732 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jpmth\"" Jul 06 17:45:23.011921 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.011902 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 17:45:23.103664 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.103579 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mgxwp" event={"ID":"a34e0be6-2c80-4817-80dd-5c4fbc90feb1","Type":"ContainerStarted","Data":"daa162d1ae70a3daf30f0bec2e8c5433764eb5aa572961cd4cf06567bdceb080"} Jul 06 17:45:23.106255 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.106222 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcrz9" event={"ID":"9e6ad0b1-1fdd-495d-9fe7-225f1112d537","Type":"ContainerStarted","Data":"5a48a1bcd8788233bf53fbe3434fd7d47e695048e06a10abf14ac0615425dea4"} Jul 06 17:45:23.123942 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.123887 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hcrz9" podStartSLOduration=34.414364541 podStartE2EDuration="50.123867875s" totalImagesPullingTime="15.709503334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:44:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:23.122834425 +0000 UTC m=+290.528367353" watchObservedRunningTime="2026-07-06 17:45:23.123867875 +0000 UTC m=+290.529400802" Jul 06 17:45:23.176893 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.176861 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-root\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177060 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.176914 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-sys\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177060 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.176996 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-wtmp\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177060 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.177036 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq25g\" (UniqueName: \"kubernetes.io/projected/74bf70e9-ba55-454c-bfee-f999658e59ac-kube-api-access-xq25g\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177228 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.177102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bf70e9-ba55-454c-bfee-f999658e59ac-metrics-client-ca\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177228 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.177162 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177228 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.177197 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177374 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.177227 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-tls\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.177374 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.177261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-textfile\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278119 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278088 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq25g\" (UniqueName: \"kubernetes.io/projected/74bf70e9-ba55-454c-bfee-f999658e59ac-kube-api-access-xq25g\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278277 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278128 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bf70e9-ba55-454c-bfee-f999658e59ac-metrics-client-ca\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278277 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278263 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278391 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278323 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278391 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278369 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-tls\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278526 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278422 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-textfile\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278526 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278517 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-root\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278634 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278560 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-sys\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278634 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:45:23.278579 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 17:45:23.278634 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278614 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-wtmp\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278774 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:45:23.278639 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-tls podName:74bf70e9-ba55-454c-bfee-f999658e59ac nodeName:}" failed. No retries permitted until 2026-07-06 17:45:23.778621655 +0000 UTC m=+291.184154565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-tls") pod "node-exporter-8pg8s" (UID: "74bf70e9-ba55-454c-bfee-f999658e59ac") : secret "node-exporter-tls" not found Jul 06 17:45:23.278838 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278784 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-wtmp\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278838 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278805 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74bf70e9-ba55-454c-bfee-f999658e59ac-metrics-client-ca\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.278944 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278910 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.279004 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.278972 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-sys\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.279074 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.279009 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74bf70e9-ba55-454c-bfee-f999658e59ac-root\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.279074 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.279060 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-textfile\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.281279 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.281253 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.286927 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.286904 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq25g\" (UniqueName: \"kubernetes.io/projected/74bf70e9-ba55-454c-bfee-f999658e59ac-kube-api-access-xq25g\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.782085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.782041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-tls\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.784905 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.784883 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74bf70e9-ba55-454c-bfee-f999658e59ac-node-exporter-tls\") pod \"node-exporter-8pg8s\" (UID: \"74bf70e9-ba55-454c-bfee-f999658e59ac\") " pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.923320 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:23.923263 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8pg8s" Jul 06 17:45:23.962063 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:23.962034 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74bf70e9_ba55_454c_bfee_f999658e59ac.slice/crio-e5575e5501fa2cc4a5e11a1eec211fdef5be572d208fdeb368ff82a004390d78 WatchSource:0}: Error finding container e5575e5501fa2cc4a5e11a1eec211fdef5be572d208fdeb368ff82a004390d78: Status 404 returned error can't find the container with id e5575e5501fa2cc4a5e11a1eec211fdef5be572d208fdeb368ff82a004390d78 Jul 06 17:45:24.112495 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:24.112401 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8pg8s" event={"ID":"74bf70e9-ba55-454c-bfee-f999658e59ac","Type":"ContainerStarted","Data":"e5575e5501fa2cc4a5e11a1eec211fdef5be572d208fdeb368ff82a004390d78"} Jul 06 17:45:24.114734 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:24.114699 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mgxwp" event={"ID":"a34e0be6-2c80-4817-80dd-5c4fbc90feb1","Type":"ContainerStarted","Data":"53828ecc75f88cda8de90da8c8ea5037dd3b7cbdfe7627cbc3dd53c1aefa6c6e"} Jul 06 17:45:24.115082 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:24.115056 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mgxwp" Jul 06 17:45:24.130141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:24.130073 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mgxwp" podStartSLOduration=252.913929353 podStartE2EDuration="4m16.130055095s" totalImagesPullingTime="3.216125742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:24.129383802 +0000 UTC m=+291.534916726" watchObservedRunningTime="2026-07-06 17:45:24.130055095 +0000 UTC m=+291.535588014" Jul 06 17:45:25.520878 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:25.520844 2558 patch_prober.go:28] interesting pod/image-registry-66c479597f-bqxb2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 17:45:25.521271 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:25.520896 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podUID="1fd82183-ab7a-4e74-97e0-447e89300893" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 17:45:26.125405 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:26.125339 2558 generic.go:356] "Generic (PLEG): container finished" podID="74bf70e9-ba55-454c-bfee-f999658e59ac" containerID="cd845609020a7dba30bda1e824759981500bd652c84d697ceea98d4d85903e58" exitCode=0 Jul 06 17:45:26.125563 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:26.125452 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8pg8s" event={"ID":"74bf70e9-ba55-454c-bfee-f999658e59ac","Type":"ContainerDied","Data":"cd845609020a7dba30bda1e824759981500bd652c84d697ceea98d4d85903e58"} Jul 06 17:45:26.982624 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:26.982594 2558 patch_prober.go:28] interesting pod/image-registry-66c479597f-bqxb2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 17:45:26.982993 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:26.982642 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podUID="1fd82183-ab7a-4e74-97e0-447e89300893" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 17:45:27.130226 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:27.130196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8pg8s" event={"ID":"74bf70e9-ba55-454c-bfee-f999658e59ac","Type":"ContainerStarted","Data":"dbca051e1965d64e9ca86fdcbf547d48a4f2f4b8a23447c9eaf1ea05245053ba"} Jul 06 17:45:27.130409 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:27.130248 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8pg8s" event={"ID":"74bf70e9-ba55-454c-bfee-f999658e59ac","Type":"ContainerStarted","Data":"20f478f27740d657c8667fb9adac2ef15bd590a38c97ae2d24d0f6fb8e738289"} Jul 06 17:45:27.147597 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:27.147557 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8pg8s" podStartSLOduration=3.243751504 podStartE2EDuration="5.147543861s" totalImagesPullingTime="1.806091927s" totalInitContainerRuntime="97.70043ms" isStatefulPod=true podCreationTimestamp="2026-07-06 17:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:27.14640728 +0000 UTC m=+294.551940206" watchObservedRunningTime="2026-07-06 17:45:27.147543861 +0000 UTC m=+294.553076785" Jul 06 17:45:27.774332 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:27.774299 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b96f44445-ccbxj"] Jul 06 17:45:29.181745 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.181709 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:45:29.205273 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.205240 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:45:29.205456 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.205362 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.207618 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.207599 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 17:45:29.207783 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.207763 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 17:45:29.208223 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.207661 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 17:45:29.208223 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.207992 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 17:45:29.208360 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.208281 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 17:45:29.208584 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.208485 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 17:45:29.209075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.208758 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 17:45:29.209582 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.209393 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 17:45:29.210020 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.210001 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 17:45:29.210132 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.210079 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 17:45:29.210242 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.210222 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 17:45:29.210541 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.210513 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-w2g2h\"" Jul 06 17:45:29.210653 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.210593 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 17:45:29.210759 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.210731 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ae3u9ge2buc6h\"" Jul 06 17:45:29.210991 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.210975 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 17:45:29.220322 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.220300 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 17:45:29.222298 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.222280 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 17:45:29.328106 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328074 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328106 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328103 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/183eab2a-a5fc-4f78-a841-2b2ef302f87b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328312 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328123 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328312 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328142 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328312 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328232 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328312 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328250 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328312 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328268 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328312 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328290 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328312 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328311 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328340 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328363 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328422 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-web-config\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328478 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78wvr\" (UniqueName: \"kubernetes.io/projected/183eab2a-a5fc-4f78-a841-2b2ef302f87b-kube-api-access-78wvr\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328496 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-config\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328528 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328564 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/183eab2a-a5fc-4f78-a841-2b2ef302f87b-config-out\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328579 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328600 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.328719 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.328615 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.429931 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.429888 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-web-config\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.429965 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78wvr\" (UniqueName: \"kubernetes.io/projected/183eab2a-a5fc-4f78-a841-2b2ef302f87b-kube-api-access-78wvr\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.429995 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-config\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430040 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430070 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/183eab2a-a5fc-4f78-a841-2b2ef302f87b-config-out\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430096 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430147 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430173 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430197 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/183eab2a-a5fc-4f78-a841-2b2ef302f87b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430226 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430253 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430285 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430341 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430864 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430368 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430864 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430403 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430864 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430460 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.430864 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.430508 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.432445 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.432360 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.433980 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.433629 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.434149 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.434124 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.434987 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.434456 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.434987 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.434605 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.434987 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.434729 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/183eab2a-a5fc-4f78-a841-2b2ef302f87b-config-out\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.434987 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.434834 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/183eab2a-a5fc-4f78-a841-2b2ef302f87b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.434987 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.434872 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-web-config\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.435314 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.435293 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.435596 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.435575 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.435681 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.435667 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.435879 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.435862 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/183eab2a-a5fc-4f78-a841-2b2ef302f87b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.436023 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.435981 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.436605 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.436252 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.437099 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.437073 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183eab2a-a5fc-4f78-a841-2b2ef302f87b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.437196 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.437178 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.437527 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.437495 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.438289 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.438265 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-config\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.438397 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.438365 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/183eab2a-a5fc-4f78-a841-2b2ef302f87b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.438651 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.438631 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78wvr\" (UniqueName: \"kubernetes.io/projected/183eab2a-a5fc-4f78-a841-2b2ef302f87b-kube-api-access-78wvr\") pod \"prometheus-k8s-0\" (UID: \"183eab2a-a5fc-4f78-a841-2b2ef302f87b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.522866 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.522737 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:29.691548 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:29.691491 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 17:45:29.694756 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:45:29.694730 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183eab2a_a5fc_4f78_a841_2b2ef302f87b.slice/crio-03dcd52410f692717a8e901a4fdac4ab3d40b371493ab19bd16c972dd3b5b23d WatchSource:0}: Error finding container 03dcd52410f692717a8e901a4fdac4ab3d40b371493ab19bd16c972dd3b5b23d: Status 404 returned error can't find the container with id 03dcd52410f692717a8e901a4fdac4ab3d40b371493ab19bd16c972dd3b5b23d Jul 06 17:45:30.069206 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:45:30.069170 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b9gd9" podUID="e7646b7b-d22f-4fab-9609-1c3beb2632e1" Jul 06 17:45:30.144301 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:30.144276 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerStarted","Data":"03dcd52410f692717a8e901a4fdac4ab3d40b371493ab19bd16c972dd3b5b23d"} Jul 06 17:45:32.152358 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:32.152322 2558 generic.go:356] "Generic (PLEG): container finished" podID="183eab2a-a5fc-4f78-a841-2b2ef302f87b" containerID="de644bb82d688ad372357e7a4fecb9856f91bd76958ccec9e76bd714ffb10a3c" exitCode=0 Jul 06 17:45:32.152776 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:32.152416 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerDied","Data":"de644bb82d688ad372357e7a4fecb9856f91bd76958ccec9e76bd714ffb10a3c"} Jul 06 17:45:32.919400 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:32.919367 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:45:32.919682 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:32.919665 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:45:32.925317 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:32.925294 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:45:32.925880 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:32.925860 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:45:32.932556 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:32.932538 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 17:45:34.122818 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:34.122767 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mgxwp" Jul 06 17:45:35.521459 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:35.521396 2558 patch_prober.go:28] interesting pod/image-registry-66c479597f-bqxb2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 17:45:35.521939 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:35.521486 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podUID="1fd82183-ab7a-4e74-97e0-447e89300893" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 17:45:35.521939 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:35.521538 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:45:35.522176 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:35.522118 2558 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"19dfec4a183932b1d5297044d1cd307de338180dc1c752b076c712052c7d019b"} pod="openshift-image-registry/image-registry-66c479597f-bqxb2" containerMessage="Container registry failed liveness probe, will be restarted" Jul 06 17:45:35.526634 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:35.526605 2558 patch_prober.go:28] interesting pod/image-registry-66c479597f-bqxb2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 17:45:35.526755 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:35.526657 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podUID="1fd82183-ab7a-4e74-97e0-447e89300893" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 17:45:36.415072 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:36.414207 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 17:45:37.174285 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:37.174242 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerStarted","Data":"4d318ca4362e833b8a0a2bf129623055957593f4864e615166cf0fa2b13c0c01"} Jul 06 17:45:37.174685 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:37.174305 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerStarted","Data":"f9f4d4f226254027d943b5a5987bfdbf3d7923be015f7486bb4435aa0c2c4ea8"} Jul 06 17:45:40.189490 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:40.189460 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerStarted","Data":"690212ad278b16be860323c1e047eab70925575dfc9b9a8dc44c2827cf4c0d03"} Jul 06 17:45:40.190011 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:40.189520 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerStarted","Data":"99e2ff69300baf7b57f633df77424c027cbf0441a34c9185518f43c7a4987632"} Jul 06 17:45:40.190011 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:40.189546 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerStarted","Data":"5ec0a1e7556fac6550ae284e2ecb4b94ee15df924f15770843749962b0b7530f"} Jul 06 17:45:40.190011 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:40.189568 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"183eab2a-a5fc-4f78-a841-2b2ef302f87b","Type":"ContainerStarted","Data":"348946ed4d9ccc3f0ba18e4d306cbd82cd4b2c36695ba6996c19bb61d8fd8d61"} Jul 06 17:45:40.214671 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:40.214635 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.977453165 podStartE2EDuration="11.214622417s" totalImagesPullingTime="8.222322377s" totalInitContainerRuntime="14.846875ms" isStatefulPod=false podCreationTimestamp="2026-07-06 17:45:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:40.213541424 +0000 UTC m=+307.619074349" watchObservedRunningTime="2026-07-06 17:45:40.214622417 +0000 UTC m=+307.620155341" Jul 06 17:45:43.069173 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:43.069141 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:45:44.523399 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:44.523371 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:45:45.526133 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:45.526110 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:45:48.216479 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:48.216452 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ef13f52-687a-47a1-afdf-a087c1b840f6" containerID="49fcca0dce78fb387572ae357b11cc346c79cfe98bc1b18c713d2c8908c4269d" exitCode=0 Jul 06 17:45:48.216804 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:48.216500 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" event={"ID":"7ef13f52-687a-47a1-afdf-a087c1b840f6","Type":"ContainerDied","Data":"49fcca0dce78fb387572ae357b11cc346c79cfe98bc1b18c713d2c8908c4269d"} Jul 06 17:45:48.216804 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:48.216743 2558 scope.go:117] "RemoveContainer" containerID="49fcca0dce78fb387572ae357b11cc346c79cfe98bc1b18c713d2c8908c4269d" Jul 06 17:45:48.725671 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:48.725635 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6575475f-sg844_64a857a2-eb58-4f29-80cf-a2462430c591/router/0.log" Jul 06 17:45:48.732528 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:48.732505 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jpln9_3a2ef3dd-ede8-498e-ae66-0941952e042a/serve-healthcheck-canary/0.log" Jul 06 17:45:49.221080 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:49.221051 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cz4mf" event={"ID":"7ef13f52-687a-47a1-afdf-a087c1b840f6","Type":"ContainerStarted","Data":"51f2fdacaf7d1b84d3c21e092d49ef0ebd0f4e12135b3e701eac6499e7317960"} Jul 06 17:45:52.795379 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:52.795342 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7b96f44445-ccbxj" podUID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" containerName="console" containerID="cri-o://b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba" gracePeriod=15 Jul 06 17:45:53.088471 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.088447 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:53.225987 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.225961 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-service-ca\") pod \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " Jul 06 17:45:53.226101 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.225990 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-oauth-config\") pod \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " Jul 06 17:45:53.226101 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.226016 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-serving-cert\") pod \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " Jul 06 17:45:53.226101 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.226056 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-config\") pod \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " Jul 06 17:45:53.226233 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.226111 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-oauth-serving-cert\") pod \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " Jul 06 17:45:53.226233 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.226140 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvgcj\" (UniqueName: \"kubernetes.io/projected/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-kube-api-access-vvgcj\") pod \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\" (UID: \"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d\") " Jul 06 17:45:53.226521 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.226475 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-service-ca" (OuterVolumeSpecName: "service-ca") pod "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" (UID: "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:45:53.226521 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.226495 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-config" (OuterVolumeSpecName: "console-config") pod "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" (UID: "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:45:53.226686 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.226569 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" (UID: "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:45:53.228043 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.228022 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" (UID: "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:45:53.228124 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.228068 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" (UID: "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:45:53.228189 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.228170 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-kube-api-access-vvgcj" (OuterVolumeSpecName: "kube-api-access-vvgcj") pod "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" (UID: "b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d"). InnerVolumeSpecName "kube-api-access-vvgcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:45:53.236342 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.236318 2558 generic.go:356] "Generic (PLEG): container finished" podID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" containerID="b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba" exitCode=0 Jul 06 17:45:53.236448 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.236346 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b96f44445-ccbxj" Jul 06 17:45:53.236448 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.236362 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b96f44445-ccbxj" event={"ID":"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d","Type":"ContainerDied","Data":"b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba"} Jul 06 17:45:53.236448 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.236405 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b96f44445-ccbxj" event={"ID":"b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d","Type":"ContainerDied","Data":"fca93f193b0205eab72df0acb3927f753cdf9357befb3b224abe857ecf1ef2b1"} Jul 06 17:45:53.236553 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.236450 2558 scope.go:117] "RemoveContainer" containerID="b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba" Jul 06 17:45:53.244183 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.244166 2558 scope.go:117] "RemoveContainer" containerID="b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba" Jul 06 17:45:53.244463 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:45:53.244422 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba\": container with ID starting with b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba not found: ID does not exist" containerID="b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba" Jul 06 17:45:53.244516 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.244472 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba"} err="failed to get container status \"b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba\": rpc error: code = NotFound desc = could not find container \"b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba\": container with ID starting with b50ab1c9885a0b587c239f7b988641b51e40655621e60766e6d17481a33de2ba not found: ID does not exist" Jul 06 17:45:53.255368 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.255345 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b96f44445-ccbxj"] Jul 06 17:45:53.258208 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.258192 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b96f44445-ccbxj"] Jul 06 17:45:53.327501 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.327458 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-config\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:45:53.327501 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.327477 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-oauth-serving-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:45:53.327501 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.327487 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvgcj\" (UniqueName: \"kubernetes.io/projected/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-kube-api-access-vvgcj\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:45:53.327501 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.327496 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-service-ca\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:45:53.327659 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.327504 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-oauth-config\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:45:53.327659 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:53.327516 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d-console-serving-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:45:55.074549 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:45:55.074492 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" path="/var/lib/kubelet/pods/b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d/volumes" Jul 06 17:46:00.552638 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:00.552485 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" podUID="1fd82183-ab7a-4e74-97e0-447e89300893" containerName="registry" containerID="cri-o://19dfec4a183932b1d5297044d1cd307de338180dc1c752b076c712052c7d019b" gracePeriod=30 Jul 06 17:46:01.269016 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:01.268978 2558 generic.go:356] "Generic (PLEG): container finished" podID="1fd82183-ab7a-4e74-97e0-447e89300893" containerID="19dfec4a183932b1d5297044d1cd307de338180dc1c752b076c712052c7d019b" exitCode=0 Jul 06 17:46:01.269212 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:01.269053 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" event={"ID":"1fd82183-ab7a-4e74-97e0-447e89300893","Type":"ContainerDied","Data":"19dfec4a183932b1d5297044d1cd307de338180dc1c752b076c712052c7d019b"} Jul 06 17:46:01.269212 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:01.269112 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" event={"ID":"1fd82183-ab7a-4e74-97e0-447e89300893","Type":"ContainerStarted","Data":"001a37c9fa40ba3dece84f137f37c422217f49be6612f226cf867f455b1c7e6a"} Jul 06 17:46:01.269314 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:01.269239 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:46:22.277800 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:22.277774 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66c479597f-bqxb2" Jul 06 17:46:29.523139 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:29.523032 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:46:29.538948 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:29.538924 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:46:30.393769 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:30.393743 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 17:46:45.867799 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:45.867764 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:46:45.870172 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:45.870148 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7646b7b-d22f-4fab-9609-1c3beb2632e1-metrics-certs\") pod \"network-metrics-daemon-b9gd9\" (UID: \"e7646b7b-d22f-4fab-9609-1c3beb2632e1\") " pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:46:46.072778 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:46.072744 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gwzng\"" Jul 06 17:46:46.081159 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:46.081137 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b9gd9" Jul 06 17:46:46.208238 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:46.208190 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b9gd9"] Jul 06 17:46:46.211103 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:46:46.211069 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7646b7b_d22f_4fab_9609_1c3beb2632e1.slice/crio-a4dbaa84d45e35ea8240c17e88b436e8f54fea2f073b63152ffc7c27ae1b32e4 WatchSource:0}: Error finding container a4dbaa84d45e35ea8240c17e88b436e8f54fea2f073b63152ffc7c27ae1b32e4: Status 404 returned error can't find the container with id a4dbaa84d45e35ea8240c17e88b436e8f54fea2f073b63152ffc7c27ae1b32e4 Jul 06 17:46:46.444700 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:46.444615 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b9gd9" event={"ID":"e7646b7b-d22f-4fab-9609-1c3beb2632e1","Type":"ContainerStarted","Data":"a4dbaa84d45e35ea8240c17e88b436e8f54fea2f073b63152ffc7c27ae1b32e4"} Jul 06 17:46:47.450749 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:47.450650 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b9gd9" event={"ID":"e7646b7b-d22f-4fab-9609-1c3beb2632e1","Type":"ContainerStarted","Data":"15f6ce21dc07b371bcc5cd4712581689dee297c7d56e25d230510c1b37cba046"} Jul 06 17:46:47.450749 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:47.450708 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b9gd9" event={"ID":"e7646b7b-d22f-4fab-9609-1c3beb2632e1","Type":"ContainerStarted","Data":"2941654669d61453a219c427a33641150da99c0729ca13ff8f2c1a9a79cf8eac"} Jul 06 17:46:47.472769 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:46:47.472720 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b9gd9" podStartSLOduration=372.645743161 podStartE2EDuration="6m13.472703168s" totalImagesPullingTime="826.960007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:46:47.471830242 +0000 UTC m=+374.877363169" watchObservedRunningTime="2026-07-06 17:46:47.472703168 +0000 UTC m=+374.878236122" Jul 06 17:49:28.137648 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.137571 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw"] Jul 06 17:49:28.138141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.137979 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" containerName="console" Jul 06 17:49:28.138141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.137994 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" containerName="console" Jul 06 17:49:28.138141 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.138047 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b06ff792-0efd-4f30-a7c2-c1fdffc5ed3d" containerName="console" Jul 06 17:49:28.141088 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.141067 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.143158 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.143116 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 17:49:28.143158 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.143151 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 17:49:28.143705 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.143688 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 17:49:28.143808 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.143717 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 17:49:28.143808 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.143688 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mmw7p\"" Jul 06 17:49:28.165349 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.165321 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw"] Jul 06 17:49:28.300313 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.300271 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f870b77-c67f-4d34-a43a-85d219f60110-tmp\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.300313 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.300319 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f870b77-c67f-4d34-a43a-85d219f60110-webhook-cert\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.300615 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.300344 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kv59\" (UniqueName: \"kubernetes.io/projected/4f870b77-c67f-4d34-a43a-85d219f60110-kube-api-access-4kv59\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.300615 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.300398 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f870b77-c67f-4d34-a43a-85d219f60110-manifests\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.300615 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.300464 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f870b77-c67f-4d34-a43a-85d219f60110-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.401463 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.401343 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f870b77-c67f-4d34-a43a-85d219f60110-manifests\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.401463 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.401392 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f870b77-c67f-4d34-a43a-85d219f60110-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.401463 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.401460 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f870b77-c67f-4d34-a43a-85d219f60110-tmp\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.401746 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.401482 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f870b77-c67f-4d34-a43a-85d219f60110-webhook-cert\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.401746 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.401498 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kv59\" (UniqueName: \"kubernetes.io/projected/4f870b77-c67f-4d34-a43a-85d219f60110-kube-api-access-4kv59\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.401852 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.401833 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4f870b77-c67f-4d34-a43a-85d219f60110-manifests\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.403805 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.403777 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f870b77-c67f-4d34-a43a-85d219f60110-tmp\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.404039 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.404022 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4f870b77-c67f-4d34-a43a-85d219f60110-webhook-cert\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.404103 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.404059 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4f870b77-c67f-4d34-a43a-85d219f60110-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.408421 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.408400 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kv59\" (UniqueName: \"kubernetes.io/projected/4f870b77-c67f-4d34-a43a-85d219f60110-kube-api-access-4kv59\") pod \"opendatahub-operator-controller-manager-74668f54-5hrqw\" (UID: \"4f870b77-c67f-4d34-a43a-85d219f60110\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.451523 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.451487 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:28.596157 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:28.595974 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw"] Jul 06 17:49:28.599157 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:49:28.599112 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f870b77_c67f_4d34_a43a_85d219f60110.slice/crio-9280c998c3eae4d0c46e59395936b794cba927691184968b03bb3ac7dc861d25 WatchSource:0}: Error finding container 9280c998c3eae4d0c46e59395936b794cba927691184968b03bb3ac7dc861d25: Status 404 returned error can't find the container with id 9280c998c3eae4d0c46e59395936b794cba927691184968b03bb3ac7dc861d25 Jul 06 17:49:29.043593 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:29.043559 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" event={"ID":"4f870b77-c67f-4d34-a43a-85d219f60110","Type":"ContainerStarted","Data":"9280c998c3eae4d0c46e59395936b794cba927691184968b03bb3ac7dc861d25"} Jul 06 17:49:32.058087 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:32.057997 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" event={"ID":"4f870b77-c67f-4d34-a43a-85d219f60110","Type":"ContainerStarted","Data":"412b4838b950d3c9022a77cc5f42999cac5704ca8cdc1b577fa467696698a5b3"} Jul 06 17:49:33.063485 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:33.063450 2558 generic.go:356] "Generic (PLEG): container finished" podID="4f870b77-c67f-4d34-a43a-85d219f60110" containerID="412b4838b950d3c9022a77cc5f42999cac5704ca8cdc1b577fa467696698a5b3" exitCode=0 Jul 06 17:49:33.063948 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:33.063499 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" event={"ID":"4f870b77-c67f-4d34-a43a-85d219f60110","Type":"ContainerDied","Data":"412b4838b950d3c9022a77cc5f42999cac5704ca8cdc1b577fa467696698a5b3"} Jul 06 17:49:34.070683 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:34.070647 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" event={"ID":"4f870b77-c67f-4d34-a43a-85d219f60110","Type":"ContainerStarted","Data":"aa69b805f16086832415cce7b9a7e0d55e96a493c24ab5112ed5942ad0815568"} Jul 06 17:49:34.071113 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:34.070802 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:34.091176 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:34.091112 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" podStartSLOduration=1.437931628 podStartE2EDuration="6.091096234s" totalImagesPullingTime="3.599208959s" totalInitContainerRuntime="1.053955647s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:34.089056894 +0000 UTC m=+541.494589820" watchObservedRunningTime="2026-07-06 17:49:34.091096234 +0000 UTC m=+541.496629160" Jul 06 17:49:45.076521 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:45.076492 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 17:49:48.105720 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.105685 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc"] Jul 06 17:49:48.120557 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.120519 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc"] Jul 06 17:49:48.120731 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.120625 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.125259 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.125234 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 17:49:48.126054 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.126032 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 17:49:48.126054 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.126040 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 17:49:48.126253 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.126243 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d8knk\"" Jul 06 17:49:48.126355 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.126335 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 17:49:48.126355 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.126350 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 17:49:48.173293 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.173258 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-manager-config\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.173502 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.173306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gtrv\" (UniqueName: \"kubernetes.io/projected/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-kube-api-access-7gtrv\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.173502 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.173457 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-metrics-cert\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.173623 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.173573 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-cert\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.274744 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.274696 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-metrics-cert\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.274911 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.274790 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-cert\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.274911 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.274844 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-manager-config\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.274911 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.274880 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gtrv\" (UniqueName: \"kubernetes.io/projected/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-kube-api-access-7gtrv\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.275630 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.275606 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-manager-config\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.277539 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.277504 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-cert\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.277864 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.277843 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-metrics-cert\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.284804 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.284759 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gtrv\" (UniqueName: \"kubernetes.io/projected/86e6ca36-58e5-4b2a-b2c4-991fed2ad46c-kube-api-access-7gtrv\") pod \"lws-controller-manager-7768b55bcf-gzbgc\" (UID: \"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c\") " pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.430717 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.430621 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:48.570309 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:48.570276 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc"] Jul 06 17:49:48.574037 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:49:48.574009 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86e6ca36_58e5_4b2a_b2c4_991fed2ad46c.slice/crio-e94d290d9a29526ed255407c789db52e5e98d2ecb03cd7492141996e6e3bcc6f WatchSource:0}: Error finding container e94d290d9a29526ed255407c789db52e5e98d2ecb03cd7492141996e6e3bcc6f: Status 404 returned error can't find the container with id e94d290d9a29526ed255407c789db52e5e98d2ecb03cd7492141996e6e3bcc6f Jul 06 17:49:49.101694 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.101664 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jssv"] Jul 06 17:49:49.106228 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.106208 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:49.108546 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.108527 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 17:49:49.119652 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.119629 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jssv"] Jul 06 17:49:49.127246 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.127222 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" event={"ID":"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c","Type":"ContainerStarted","Data":"e94d290d9a29526ed255407c789db52e5e98d2ecb03cd7492141996e6e3bcc6f"} Jul 06 17:49:49.182868 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.182830 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhnnm\" (UniqueName: \"kubernetes.io/projected/3dca3e2f-9412-4154-9717-9d1c98560e2b-kube-api-access-rhnnm\") pod \"model-serving-api-54d89bd5fc-9jssv\" (UID: \"3dca3e2f-9412-4154-9717-9d1c98560e2b\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:49.183044 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.182883 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dca3e2f-9412-4154-9717-9d1c98560e2b-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jssv\" (UID: \"3dca3e2f-9412-4154-9717-9d1c98560e2b\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:49.283404 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.283359 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhnnm\" (UniqueName: \"kubernetes.io/projected/3dca3e2f-9412-4154-9717-9d1c98560e2b-kube-api-access-rhnnm\") pod \"model-serving-api-54d89bd5fc-9jssv\" (UID: \"3dca3e2f-9412-4154-9717-9d1c98560e2b\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:49.283594 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.283459 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dca3e2f-9412-4154-9717-9d1c98560e2b-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jssv\" (UID: \"3dca3e2f-9412-4154-9717-9d1c98560e2b\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:49.283658 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:49:49.283618 2558 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 06 17:49:49.283706 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:49:49.283691 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dca3e2f-9412-4154-9717-9d1c98560e2b-tls-certs podName:3dca3e2f-9412-4154-9717-9d1c98560e2b nodeName:}" failed. No retries permitted until 2026-07-06 17:49:49.783675039 +0000 UTC m=+557.189207942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3dca3e2f-9412-4154-9717-9d1c98560e2b-tls-certs") pod "model-serving-api-54d89bd5fc-9jssv" (UID: "3dca3e2f-9412-4154-9717-9d1c98560e2b") : secret "model-serving-api-tls" not found Jul 06 17:49:49.291946 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.291917 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhnnm\" (UniqueName: \"kubernetes.io/projected/3dca3e2f-9412-4154-9717-9d1c98560e2b-kube-api-access-rhnnm\") pod \"model-serving-api-54d89bd5fc-9jssv\" (UID: \"3dca3e2f-9412-4154-9717-9d1c98560e2b\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:49.790274 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.790231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dca3e2f-9412-4154-9717-9d1c98560e2b-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jssv\" (UID: \"3dca3e2f-9412-4154-9717-9d1c98560e2b\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:49.793152 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:49.793122 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dca3e2f-9412-4154-9717-9d1c98560e2b-tls-certs\") pod \"model-serving-api-54d89bd5fc-9jssv\" (UID: \"3dca3e2f-9412-4154-9717-9d1c98560e2b\") " pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:50.016686 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:50.016644 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:51.168731 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:51.168527 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9jssv"] Jul 06 17:49:51.172486 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:49:51.172450 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dca3e2f_9412_4154_9717_9d1c98560e2b.slice/crio-3cb51c0e1dca62c96e9863a210e4bc094b168701407e5aa325e38bf67cd7acc2 WatchSource:0}: Error finding container 3cb51c0e1dca62c96e9863a210e4bc094b168701407e5aa325e38bf67cd7acc2: Status 404 returned error can't find the container with id 3cb51c0e1dca62c96e9863a210e4bc094b168701407e5aa325e38bf67cd7acc2 Jul 06 17:49:52.143695 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:52.143646 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" event={"ID":"86e6ca36-58e5-4b2a-b2c4-991fed2ad46c","Type":"ContainerStarted","Data":"6d87ba219641cbcf7695e6060073ff0ce0f7e369a3ffb1303c491e8d01461e56"} Jul 06 17:49:52.143895 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:52.143754 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:49:52.145354 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:52.145310 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" event={"ID":"3dca3e2f-9412-4154-9717-9d1c98560e2b","Type":"ContainerStarted","Data":"3cb51c0e1dca62c96e9863a210e4bc094b168701407e5aa325e38bf67cd7acc2"} Jul 06 17:49:52.160254 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:52.160203 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" podStartSLOduration=1.6383840539999999 podStartE2EDuration="4.160186187s" totalImagesPullingTime="2.521802133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:52.15831058 +0000 UTC m=+559.563843510" watchObservedRunningTime="2026-07-06 17:49:52.160186187 +0000 UTC m=+559.565719112" Jul 06 17:49:54.156327 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.156284 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" event={"ID":"3dca3e2f-9412-4154-9717-9d1c98560e2b","Type":"ContainerStarted","Data":"0afb77270beb385a7acad917aa32ba1c49cc6a8bab8e70de74e00cc7fc241c6e"} Jul 06 17:49:54.156776 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.156405 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:49:54.171604 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.171556 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" podStartSLOduration=3.18311698 podStartE2EDuration="5.171539822s" totalImagesPullingTime="1.988422842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:54.170274643 +0000 UTC m=+561.575807570" watchObservedRunningTime="2026-07-06 17:49:54.171539822 +0000 UTC m=+561.577072747" Jul 06 17:49:54.539461 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.539409 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj"] Jul 06 17:49:54.543228 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.543186 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.545372 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.545346 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qgwvg\"" Jul 06 17:49:54.545535 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.545346 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 17:49:54.551927 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.551873 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj"] Jul 06 17:49:54.598334 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.598297 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m"] Jul 06 17:49:54.601566 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.601542 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.615330 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.615298 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m"] Jul 06 17:49:54.636660 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636622 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.636660 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636658 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.636907 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636683 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.636907 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636798 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f738c98-8361-4755-8adc-30c53ba0d0df-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.636907 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636844 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.636907 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636900 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.637085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636944 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.637085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.636993 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.637085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637023 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.637085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.637085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637061 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.637085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637081 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8bws\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-kube-api-access-w8bws\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.637422 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637100 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cnj5\" (UniqueName: \"kubernetes.io/projected/6f738c98-8361-4755-8adc-30c53ba0d0df-kube-api-access-2cnj5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.637422 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637123 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.637422 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637147 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.637422 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637163 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.637422 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637226 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.637422 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.637262 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.737860 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.737825 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.737860 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.737868 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738145 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.737895 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738145 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738019 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738145 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738075 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.738145 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738099 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738145 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738118 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.738145 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738140 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f738c98-8361-4755-8adc-30c53ba0d0df-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738457 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738308 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.738457 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738320 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738457 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738366 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738457 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738395 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738457 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738395 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738706 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738471 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738706 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.738706 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738558 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.738865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738788 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.738865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738801 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.738865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738836 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.738865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738861 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.739064 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738889 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.739133 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.738948 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8bws\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-kube-api-access-w8bws\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.739190 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.739164 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cnj5\" (UniqueName: \"kubernetes.io/projected/6f738c98-8361-4755-8adc-30c53ba0d0df-kube-api-access-2cnj5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.739284 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.739055 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f738c98-8361-4755-8adc-30c53ba0d0df-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.739284 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.739119 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.739284 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.739087 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.739489 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.739311 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.739694 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.739669 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.740758 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.740733 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.740911 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.740893 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.741135 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.741113 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.741387 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.741372 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.750763 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.750682 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f738c98-8361-4755-8adc-30c53ba0d0df-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.751054 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.751019 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cnj5\" (UniqueName: \"kubernetes.io/projected/6f738c98-8361-4755-8adc-30c53ba0d0df-kube-api-access-2cnj5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4b8v7m\" (UID: \"6f738c98-8361-4755-8adc-30c53ba0d0df\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:54.751170 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.751117 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8bws\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-kube-api-access-w8bws\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.751260 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.751242 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.855846 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.855753 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:54.912349 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:54.912314 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:55.025592 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:55.025536 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj"] Jul 06 17:49:55.028213 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:49:55.028186 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0bcaa0_dbad_4f30_8bf5_9b3d1cfca60d.slice/crio-e24884e547f7c53ed69034f6d1ef9d0f9cdeec7a44baa3e198c8e276ad78c90c WatchSource:0}: Error finding container e24884e547f7c53ed69034f6d1ef9d0f9cdeec7a44baa3e198c8e276ad78c90c: Status 404 returned error can't find the container with id e24884e547f7c53ed69034f6d1ef9d0f9cdeec7a44baa3e198c8e276ad78c90c Jul 06 17:49:55.083635 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:55.083598 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m"] Jul 06 17:49:55.087047 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:49:55.087008 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f738c98_8361_4755_8adc_30c53ba0d0df.slice/crio-072a3c2931c5dcbf69aa1de60b80b6dca789af1610e4c45f1199fd38e4f71e36 WatchSource:0}: Error finding container 072a3c2931c5dcbf69aa1de60b80b6dca789af1610e4c45f1199fd38e4f71e36: Status 404 returned error can't find the container with id 072a3c2931c5dcbf69aa1de60b80b6dca789af1610e4c45f1199fd38e4f71e36 Jul 06 17:49:55.163142 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:55.163052 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" event={"ID":"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d","Type":"ContainerStarted","Data":"e24884e547f7c53ed69034f6d1ef9d0f9cdeec7a44baa3e198c8e276ad78c90c"} Jul 06 17:49:55.164217 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:55.164192 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" event={"ID":"6f738c98-8361-4755-8adc-30c53ba0d0df","Type":"ContainerStarted","Data":"072a3c2931c5dcbf69aa1de60b80b6dca789af1610e4c45f1199fd38e4f71e36"} Jul 06 17:49:56.655571 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.655477 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92"] Jul 06 17:49:56.659075 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.659046 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.661896 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.661471 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 17:49:56.661896 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.661494 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gvj69\"" Jul 06 17:49:56.661896 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.661471 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 17:49:56.661896 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.661700 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 17:49:56.670539 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.670514 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92"] Jul 06 17:49:56.758836 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.758798 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.759009 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.758859 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbff2003-37be-4e2f-b7bf-4808a220acd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.759009 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.758946 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.759009 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.759001 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.759165 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.759038 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.759165 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.759100 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.759165 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.759121 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf7bk\" (UniqueName: \"kubernetes.io/projected/fbff2003-37be-4e2f-b7bf-4808a220acd0-kube-api-access-rf7bk\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.860020 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.859985 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.860195 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.860040 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.860195 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.860097 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.860195 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.860124 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf7bk\" (UniqueName: \"kubernetes.io/projected/fbff2003-37be-4e2f-b7bf-4808a220acd0-kube-api-access-rf7bk\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.860195 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.860166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.860561 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.860205 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbff2003-37be-4e2f-b7bf-4808a220acd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.860561 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.860233 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.861241 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.861187 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.863241 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.863207 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.863335 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.863279 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.863502 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.863474 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbff2003-37be-4e2f-b7bf-4808a220acd0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.863762 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.863705 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.876516 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.876467 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf7bk\" (UniqueName: \"kubernetes.io/projected/fbff2003-37be-4e2f-b7bf-4808a220acd0-kube-api-access-rf7bk\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.878381 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.878357 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbff2003-37be-4e2f-b7bf-4808a220acd0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-nhx92\" (UID: \"fbff2003-37be-4e2f-b7bf-4808a220acd0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:56.973346 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:56.973256 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:49:57.151555 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:57.151525 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92"] Jul 06 17:49:58.145839 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:49:58.145797 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbff2003_37be_4e2f_b7bf_4808a220acd0.slice/crio-68aa7173257fcc7c000916bcc1772c42f09c949ae8539b0e1a291a37ffc9ab21 WatchSource:0}: Error finding container 68aa7173257fcc7c000916bcc1772c42f09c949ae8539b0e1a291a37ffc9ab21: Status 404 returned error can't find the container with id 68aa7173257fcc7c000916bcc1772c42f09c949ae8539b0e1a291a37ffc9ab21 Jul 06 17:49:58.182082 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:58.182052 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" event={"ID":"fbff2003-37be-4e2f-b7bf-4808a220acd0","Type":"ContainerStarted","Data":"68aa7173257fcc7c000916bcc1772c42f09c949ae8539b0e1a291a37ffc9ab21"} Jul 06 17:49:58.230513 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:58.230466 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:49:58.230613 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:58.230549 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:49:58.230613 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:58.230579 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:49:58.237358 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:58.237333 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:49:58.237408 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:58.237394 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:49:58.237467 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:58.237422 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:49:59.189877 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.189832 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" event={"ID":"6f738c98-8361-4755-8adc-30c53ba0d0df","Type":"ContainerStarted","Data":"b6f999d02729ec9a2fc3df479c4a8e8fd68768933d47400fdc8dcef3221473a9"} Jul 06 17:49:59.191666 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.191635 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" event={"ID":"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d","Type":"ContainerStarted","Data":"7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92"} Jul 06 17:49:59.227383 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.227328 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" podStartSLOduration=2.080302 podStartE2EDuration="5.227305569s" totalImagesPullingTime="3.147003569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:59.224728112 +0000 UTC m=+566.630261037" watchObservedRunningTime="2026-07-06 17:49:59.227305569 +0000 UTC m=+566.632838496" Jul 06 17:49:59.268496 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.268417 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" podStartSLOduration=2.0688894 podStartE2EDuration="5.26839458s" totalImagesPullingTime="3.19950518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:59.266355297 +0000 UTC m=+566.671888222" watchObservedRunningTime="2026-07-06 17:49:59.26839458 +0000 UTC m=+566.673927505" Jul 06 17:49:59.856321 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.856288 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:49:59.858050 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.858021 2558 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body= Jul 06 17:49:59.858182 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.858088 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" Jul 06 17:49:59.913357 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.913311 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:49:59.918681 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:49:59.918655 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:50:00.197775 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:00.197698 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:50:00.198890 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:00.198867 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4b8v7m" Jul 06 17:50:00.257277 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:00.257197 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj"] Jul 06 17:50:00.486492 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:00.486456 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:50:00.486568 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:00.486523 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:50:00.856808 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:00.856773 2558 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body= Jul 06 17:50:00.856989 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:00.856831 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" Jul 06 17:50:01.204989 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:01.204859 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" event={"ID":"fbff2003-37be-4e2f-b7bf-4808a220acd0","Type":"ContainerStarted","Data":"8282a448b7badb78009e2fb9dadf6a39ee356151d356ff3a0f0ca513ffb43cae"} Jul 06 17:50:01.205483 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:01.205140 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:50:01.207076 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:01.207045 2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-nhx92 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 17:50:01.207221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:01.207110 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" podUID="fbff2003-37be-4e2f-b7bf-4808a220acd0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 17:50:01.226907 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:01.226845 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" podStartSLOduration=2.88863371 podStartE2EDuration="5.226822751s" totalImagesPullingTime="2.338189041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:50:01.223863273 +0000 UTC m=+568.629396201" watchObservedRunningTime="2026-07-06 17:50:01.226822751 +0000 UTC m=+568.632355678" Jul 06 17:50:01.856147 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:01.856110 2558 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body= Jul 06 17:50:01.856339 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:01.856169 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" Jul 06 17:50:02.209218 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:02.209109 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerName="istio-proxy" containerID="cri-o://7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92" gracePeriod=30 Jul 06 17:50:02.209675 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:02.209592 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-nhx92" Jul 06 17:50:03.152402 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:03.152370 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7768b55bcf-gzbgc" Jul 06 17:50:05.168906 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:05.168874 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9jssv" Jul 06 17:50:07.456750 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.456723 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:50:07.568395 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568358 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-credential-socket\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568606 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568418 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-socket\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568606 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568485 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8bws\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-kube-api-access-w8bws\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568606 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568511 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-certs\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568606 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568545 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-podinfo\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568606 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568598 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-data\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568621 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istiod-ca-cert\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568653 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-token\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568667 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:50:07.568865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568712 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-envoy\") pod \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\" (UID: \"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d\") " Jul 06 17:50:07.568865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568731 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:50:07.568865 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568843 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:50:07.569164 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568860 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:50:07.569164 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.568995 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 17:50:07.569164 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.569002 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.569164 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.569034 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.569164 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.569054 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.569164 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.569069 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.571108 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.571065 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-kube-api-access-w8bws" (OuterVolumeSpecName: "kube-api-access-w8bws") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "kube-api-access-w8bws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:50:07.571247 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.571155 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:50:07.571314 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.571240 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:50:07.571314 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.571268 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" (UID: "0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 17:50:07.669750 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.669709 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.669750 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.669749 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.669981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.669763 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.669981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.669775 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:07.669981 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:07.669788 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8bws\" (UniqueName: \"kubernetes.io/projected/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d-kube-api-access-w8bws\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 17:50:08.235926 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.235885 2558 generic.go:356] "Generic (PLEG): container finished" podID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerID="7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92" exitCode=0 Jul 06 17:50:08.236095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.235934 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" Jul 06 17:50:08.236095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.235940 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" event={"ID":"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d","Type":"ContainerDied","Data":"7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92"} Jul 06 17:50:08.236095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.235992 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj" event={"ID":"0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d","Type":"ContainerDied","Data":"e24884e547f7c53ed69034f6d1ef9d0f9cdeec7a44baa3e198c8e276ad78c90c"} Jul 06 17:50:08.236095 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.236023 2558 scope.go:117] "RemoveContainer" containerID="7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92" Jul 06 17:50:08.245033 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.245016 2558 scope.go:117] "RemoveContainer" containerID="7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92" Jul 06 17:50:08.245309 ip-10-0-140-115 kubenswrapper[2558]: E0706 17:50:08.245290 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92\": container with ID starting with 7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92 not found: ID does not exist" containerID="7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92" Jul 06 17:50:08.245352 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.245318 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92"} err="failed to get container status \"7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92\": rpc error: code = NotFound desc = could not find container \"7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92\": container with ID starting with 7452f3f746c72caaa9ff8e44911144fab62297374b9e233017d3ba6e9ebf7a92 not found: ID does not exist" Jul 06 17:50:08.256248 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.256219 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj"] Jul 06 17:50:08.259280 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:08.259261 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-6dlwj"] Jul 06 17:50:09.070589 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:09.070557 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" path="/var/lib/kubelet/pods/0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d/volumes" Jul 06 17:50:32.946616 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:32.946593 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:50:32.948553 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:32.948530 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:50:32.951151 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:32.951118 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:50:32.952855 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:50:32.952837 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:51:55.569256 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.569227 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5457b7966d-59cbr"] Jul 06 17:51:55.570944 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.570915 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerName="istio-proxy" Jul 06 17:51:55.571134 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.571111 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerName="istio-proxy" Jul 06 17:51:55.571269 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.571227 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0bcaa0-dbad-4f30-8bf5-9b3d1cfca60d" containerName="istio-proxy" Jul 06 17:51:55.574460 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.574440 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.577171 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.576948 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 17:51:55.577171 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.576974 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 17:51:55.577742 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.577526 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 17:51:55.577951 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.577932 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kbbzg\"" Jul 06 17:51:55.581203 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.579820 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 17:51:55.581203 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.580042 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 17:51:55.584439 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.584406 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 17:51:55.585178 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.585155 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5457b7966d-59cbr"] Jul 06 17:51:55.637716 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.637691 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-trusted-ca-bundle\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.637858 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.637731 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28983549-fc45-4d92-9f2c-c738fcf35d82-console-serving-cert\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.637858 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.637751 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-console-config\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.637858 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.637825 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28983549-fc45-4d92-9f2c-c738fcf35d82-console-oauth-config\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.637973 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.637880 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-oauth-serving-cert\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.637973 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.637927 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-service-ca\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.637973 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.637946 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbmbp\" (UniqueName: \"kubernetes.io/projected/28983549-fc45-4d92-9f2c-c738fcf35d82-kube-api-access-sbmbp\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.738695 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.738662 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-service-ca\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.738847 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.738709 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbmbp\" (UniqueName: \"kubernetes.io/projected/28983549-fc45-4d92-9f2c-c738fcf35d82-kube-api-access-sbmbp\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.738847 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.738749 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-trusted-ca-bundle\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.738847 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.738775 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28983549-fc45-4d92-9f2c-c738fcf35d82-console-serving-cert\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.738847 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.738794 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-console-config\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.739043 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.738860 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28983549-fc45-4d92-9f2c-c738fcf35d82-console-oauth-config\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.739043 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.738908 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-oauth-serving-cert\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.739511 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.739484 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-service-ca\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.739750 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.739517 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-console-config\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.739750 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.739589 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-oauth-serving-cert\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.739873 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.739648 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28983549-fc45-4d92-9f2c-c738fcf35d82-trusted-ca-bundle\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.741212 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.741192 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28983549-fc45-4d92-9f2c-c738fcf35d82-console-oauth-config\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.741440 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.741408 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28983549-fc45-4d92-9f2c-c738fcf35d82-console-serving-cert\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.746965 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.746943 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbmbp\" (UniqueName: \"kubernetes.io/projected/28983549-fc45-4d92-9f2c-c738fcf35d82-kube-api-access-sbmbp\") pod \"console-5457b7966d-59cbr\" (UID: \"28983549-fc45-4d92-9f2c-c738fcf35d82\") " pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:55.888381 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:55.888307 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:51:56.023532 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:56.023510 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5457b7966d-59cbr"] Jul 06 17:51:56.025584 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:51:56.025557 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28983549_fc45_4d92_9f2c_c738fcf35d82.slice/crio-3d30118454d1e7d15a1d35e2b6bfdac8fff250852d29820be42f2a417b9f926c WatchSource:0}: Error finding container 3d30118454d1e7d15a1d35e2b6bfdac8fff250852d29820be42f2a417b9f926c: Status 404 returned error can't find the container with id 3d30118454d1e7d15a1d35e2b6bfdac8fff250852d29820be42f2a417b9f926c Jul 06 17:51:56.706120 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:56.706081 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5457b7966d-59cbr" event={"ID":"28983549-fc45-4d92-9f2c-c738fcf35d82","Type":"ContainerStarted","Data":"824486f03a786519497042a73908a28df1d69c1b2876558f0672ea92e7365725"} Jul 06 17:51:56.706516 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:56.706140 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5457b7966d-59cbr" event={"ID":"28983549-fc45-4d92-9f2c-c738fcf35d82","Type":"ContainerStarted","Data":"3d30118454d1e7d15a1d35e2b6bfdac8fff250852d29820be42f2a417b9f926c"} Jul 06 17:51:56.724053 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:51:56.724004 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5457b7966d-59cbr" podStartSLOduration=1.723989612 podStartE2EDuration="1.723989612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:56.722697293 +0000 UTC m=+684.128230220" watchObservedRunningTime="2026-07-06 17:51:56.723989612 +0000 UTC m=+684.129522537" Jul 06 17:52:05.888909 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:05.888875 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:52:05.889287 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:05.888934 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:52:05.894356 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:05.894334 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:52:06.754012 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:06.753984 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5457b7966d-59cbr" Jul 06 17:52:31.283286 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.283201 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz"] Jul 06 17:52:31.286636 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.286616 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.288593 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.288564 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9cbj8\"" Jul 06 17:52:31.295164 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.295139 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz"] Jul 06 17:52:31.336683 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336655 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.336809 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336690 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.336809 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336721 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.336809 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336743 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.336809 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336782 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.337026 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336884 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.337026 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336920 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxzj2\" (UniqueName: \"kubernetes.io/projected/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-kube-api-access-mxzj2\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.337026 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.336951 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.337026 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.337007 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437461 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437641 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437472 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxzj2\" (UniqueName: \"kubernetes.io/projected/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-kube-api-access-mxzj2\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437641 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437506 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437641 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437550 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437641 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437641 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437629 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437978 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437670 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437978 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437812 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437978 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437931 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.437978 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.437970 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.438185 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.438043 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.438185 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.438117 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.438313 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.438290 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.438830 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.438808 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.440092 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.440070 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.440227 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.440183 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.444626 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.444600 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.444769 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.444749 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxzj2\" (UniqueName: \"kubernetes.io/projected/012664e2-79dd-41c6-80d3-e3a4c9ccb24e-kube-api-access-mxzj2\") pod \"maas-default-gateway-openshift-default-56654dcf7-cksjz\" (UID: \"012664e2-79dd-41c6-80d3-e3a4c9ccb24e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.598315 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.598237 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:31.934325 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.934305 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz"] Jul 06 17:52:31.936151 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:52:31.936125 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012664e2_79dd_41c6_80d3_e3a4c9ccb24e.slice/crio-c71d181feba8586324dd9d879346d4b8f98739858b65e2c14e74584db4fc6868 WatchSource:0}: Error finding container c71d181feba8586324dd9d879346d4b8f98739858b65e2c14e74584db4fc6868: Status 404 returned error can't find the container with id c71d181feba8586324dd9d879346d4b8f98739858b65e2c14e74584db4fc6868 Jul 06 17:52:31.939221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.938783 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:52:31.939221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.938944 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:52:31.939221 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:31.939039 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:52:32.863211 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:32.863178 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" event={"ID":"012664e2-79dd-41c6-80d3-e3a4c9ccb24e","Type":"ContainerStarted","Data":"bc80c27d47357cd32c80fc71150a2cd8d29267e202ef35f2c79da1406acbf91d"} Jul 06 17:52:32.863603 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:32.863237 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" event={"ID":"012664e2-79dd-41c6-80d3-e3a4c9ccb24e","Type":"ContainerStarted","Data":"c71d181feba8586324dd9d879346d4b8f98739858b65e2c14e74584db4fc6868"} Jul 06 17:52:32.883276 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:32.883232 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" podStartSLOduration=1.8832183900000001 podStartE2EDuration="1.88321839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:52:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:52:32.881195326 +0000 UTC m=+720.286728249" watchObservedRunningTime="2026-07-06 17:52:32.88321839 +0000 UTC m=+720.288751394" Jul 06 17:52:33.598528 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:33.598494 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:33.603226 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:33.603202 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:33.868615 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:33.868546 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:52:33.869231 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:52:33.869216 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:53:58.256483 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:58.256422 2558 generic.go:356] "Generic (PLEG): container finished" podID="012664e2-79dd-41c6-80d3-e3a4c9ccb24e" containerID="bc80c27d47357cd32c80fc71150a2cd8d29267e202ef35f2c79da1406acbf91d" exitCode=0 Jul 06 17:53:58.256887 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:58.256499 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" event={"ID":"012664e2-79dd-41c6-80d3-e3a4c9ccb24e","Type":"ContainerDied","Data":"bc80c27d47357cd32c80fc71150a2cd8d29267e202ef35f2c79da1406acbf91d"} Jul 06 17:53:58.256887 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:58.256867 2558 scope.go:117] "RemoveContainer" containerID="bc80c27d47357cd32c80fc71150a2cd8d29267e202ef35f2c79da1406acbf91d" Jul 06 17:53:58.257842 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:58.257812 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:53:58.257895 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:58.257883 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:53:58.257943 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:58.257923 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 17:53:59.263237 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:59.263206 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" event={"ID":"012664e2-79dd-41c6-80d3-e3a4c9ccb24e","Type":"ContainerStarted","Data":"57134ca92c7279920241601a36b3a065d8efefa93f5987e2d4d75dc20c1425d7"} Jul 06 17:53:59.598727 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:59.598634 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:53:59.599759 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:59.599736 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cksjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 06 17:53:59.599863 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:53:59.599782 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" podUID="012664e2-79dd-41c6-80d3-e3a4c9ccb24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 06 17:54:00.598841 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:00.598808 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cksjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 06 17:54:00.599288 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:00.598867 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" podUID="012664e2-79dd-41c6-80d3-e3a4c9ccb24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 06 17:54:01.599131 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:01.599095 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cksjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 06 17:54:01.599534 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:01.599147 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" podUID="012664e2-79dd-41c6-80d3-e3a4c9ccb24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 06 17:54:02.598655 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:02.598615 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cksjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 06 17:54:02.598827 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:02.598685 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" podUID="012664e2-79dd-41c6-80d3-e3a4c9ccb24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 06 17:54:03.602249 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:03.602212 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:54:03.602727 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:03.602616 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:54:03.603314 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:03.603294 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cksjz" Jul 06 17:54:30.080755 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.080717 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8"] Jul 06 17:54:30.084591 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.084570 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.086749 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.086724 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 17:54:30.087329 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.087307 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kkmmc\"" Jul 06 17:54:30.087411 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.087307 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 17:54:30.087411 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.087313 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 17:54:30.092311 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.091880 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8"] Jul 06 17:54:30.123613 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.123584 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.123767 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.123635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mglwc\" (UniqueName: \"kubernetes.io/projected/75ac95be-f892-437d-aa56-10b2659f51bc-kube-api-access-mglwc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.123767 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.123721 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.123842 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.123771 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.123842 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.123799 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ac95be-f892-437d-aa56-10b2659f51bc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.123905 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.123853 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.224605 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.224568 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mglwc\" (UniqueName: \"kubernetes.io/projected/75ac95be-f892-437d-aa56-10b2659f51bc-kube-api-access-mglwc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.224788 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.224619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.224788 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.224645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.224788 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.224665 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ac95be-f892-437d-aa56-10b2659f51bc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.224788 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.224702 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.224788 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.224727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.225083 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.225059 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.225143 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.225113 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.225200 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.225182 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.226898 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.226878 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ac95be-f892-437d-aa56-10b2659f51bc-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.227177 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.227162 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ac95be-f892-437d-aa56-10b2659f51bc-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.231867 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.231842 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mglwc\" (UniqueName: \"kubernetes.io/projected/75ac95be-f892-437d-aa56-10b2659f51bc-kube-api-access-mglwc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8\" (UID: \"75ac95be-f892-437d-aa56-10b2659f51bc\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.396158 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.396076 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:30.525618 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.525582 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8"] Jul 06 17:54:30.527938 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:54:30.527911 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ac95be_f892_437d_aa56_10b2659f51bc.slice/crio-3982212346bf18447b95719c9aa50f659a66e6c6d9a2b4adb6c1d88d0774d496 WatchSource:0}: Error finding container 3982212346bf18447b95719c9aa50f659a66e6c6d9a2b4adb6c1d88d0774d496: Status 404 returned error can't find the container with id 3982212346bf18447b95719c9aa50f659a66e6c6d9a2b4adb6c1d88d0774d496 Jul 06 17:54:30.529181 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:30.529162 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 17:54:31.406352 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:31.406309 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" event={"ID":"75ac95be-f892-437d-aa56-10b2659f51bc","Type":"ContainerStarted","Data":"3982212346bf18447b95719c9aa50f659a66e6c6d9a2b4adb6c1d88d0774d496"} Jul 06 17:54:32.413333 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:32.413299 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" event={"ID":"75ac95be-f892-437d-aa56-10b2659f51bc","Type":"ContainerStarted","Data":"64e48a1a9f1566cc020d8f1711bcde405435c14cccfa6ae37b70b22aab55208a"} Jul 06 17:54:32.429996 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:32.429943 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" podStartSLOduration=1.344427706 podStartE2EDuration="2.429928324s" totalImagesPullingTime="1.085500618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:54:32.427759259 +0000 UTC m=+839.833292187" watchObservedRunningTime="2026-07-06 17:54:32.429928324 +0000 UTC m=+839.835461229" Jul 06 17:54:40.396997 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:40.396959 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:40.397339 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:40.397009 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:40.409308 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:40.409285 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:40.461580 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:40.461559 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8" Jul 06 17:54:47.593844 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.593803 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8"] Jul 06 17:54:47.600559 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.600536 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.602644 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.602619 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 17:54:47.605568 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.605548 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8"] Jul 06 17:54:47.683993 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.683967 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfb9w\" (UniqueName: \"kubernetes.io/projected/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-kube-api-access-cfb9w\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.684085 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.684019 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.684131 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.684099 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.684131 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.684124 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.684194 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.684174 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.684228 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.684190 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.784764 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.784735 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.784885 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.784769 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.784885 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.784827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.784885 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.784850 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.785029 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.784893 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfb9w\" (UniqueName: \"kubernetes.io/projected/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-kube-api-access-cfb9w\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.785029 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.784954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.785192 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.785166 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.785386 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.785262 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.785386 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.785310 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.787175 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.787155 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.787318 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.787300 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.791763 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.791741 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfb9w\" (UniqueName: \"kubernetes.io/projected/28da37ec-bdae-4ff8-8036-27ce2c9a54e0-kube-api-access-cfb9w\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pwng8\" (UID: \"28da37ec-bdae-4ff8-8036-27ce2c9a54e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:47.912258 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:47.912197 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:48.048407 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:48.048386 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8"] Jul 06 17:54:48.050617 ip-10-0-140-115 kubenswrapper[2558]: W0706 17:54:48.050591 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28da37ec_bdae_4ff8_8036_27ce2c9a54e0.slice/crio-80e76c1d8bbf75f9cffcc2db09ee3bed5ffc9107240a67f92bc85dfac4bc163b WatchSource:0}: Error finding container 80e76c1d8bbf75f9cffcc2db09ee3bed5ffc9107240a67f92bc85dfac4bc163b: Status 404 returned error can't find the container with id 80e76c1d8bbf75f9cffcc2db09ee3bed5ffc9107240a67f92bc85dfac4bc163b Jul 06 17:54:48.490464 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:48.490411 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" event={"ID":"28da37ec-bdae-4ff8-8036-27ce2c9a54e0","Type":"ContainerStarted","Data":"34222e0ed2aa277b9a3bbc8f4652da6559b89342929fb2a0bdf0e5ff02e5cc15"} Jul 06 17:54:48.490598 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:48.490485 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" event={"ID":"28da37ec-bdae-4ff8-8036-27ce2c9a54e0","Type":"ContainerStarted","Data":"80e76c1d8bbf75f9cffcc2db09ee3bed5ffc9107240a67f92bc85dfac4bc163b"} Jul 06 17:54:48.506030 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:48.505952 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" podStartSLOduration=1.309472394 podStartE2EDuration="1.505936702s" totalImagesPullingTime="196.464308ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:54:48.504293622 +0000 UTC m=+855.909826550" watchObservedRunningTime="2026-07-06 17:54:48.505936702 +0000 UTC m=+855.911469628" Jul 06 17:54:57.912526 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:57.912489 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:57.912933 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:57.912549 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:57.924597 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:57.924570 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:54:58.547440 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:54:58.547407 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pwng8" Jul 06 17:55:32.981776 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:55:32.981703 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:55:32.981776 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:55:32.981736 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 17:55:32.986263 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:55:32.986239 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 17:55:32.986454 ip-10-0-140-115 kubenswrapper[2558]: I0706 17:55:32.986294 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:00:33.012299 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:00:33.012270 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:00:33.014001 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:00:33.013981 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:00:33.016628 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:00:33.016611 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:00:33.018352 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:00:33.018336 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:05:33.042525 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:05:33.042501 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:05:33.044391 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:05:33.044362 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:05:33.047262 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:05:33.047240 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:05:33.048949 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:05:33.048930 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:10:11.825147 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.825111 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk"] Jul 06 18:10:11.828592 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.828572 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.830898 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.830876 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-12b475ae-openshift-default-dockercfg-xq6zx\"" Jul 06 18:10:11.845313 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.845290 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk"] Jul 06 18:10:11.908658 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908618 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-socket\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.908825 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908743 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-credential-socket\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.908825 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908790 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20c163a5-bae4-490f-9629-b550a1458f1f-istio-podinfo\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.908825 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908814 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-envoy\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.908980 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908828 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-certs\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.908980 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908916 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-istio-token\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.908980 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908949 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20c163a5-bae4-490f-9629-b550a1458f1f-istiod-ca-cert\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.909117 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.908983 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-data\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:11.909117 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:11.909015 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kdh8\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-kube-api-access-5kdh8\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010243 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010208 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-data\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010243 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010250 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kdh8\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-kube-api-access-5kdh8\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010275 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-socket\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010346 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-credential-socket\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010385 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20c163a5-bae4-490f-9629-b550a1458f1f-istio-podinfo\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010414 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-envoy\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010453 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-certs\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-istio-token\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20c163a5-bae4-490f-9629-b550a1458f1f-istiod-ca-cert\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010893 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010709 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-data\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010893 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010732 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-credential-socket\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.010893 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010790 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-socket\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.011003 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.010923 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-certs\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.011190 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.011173 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20c163a5-bae4-490f-9629-b550a1458f1f-istiod-ca-cert\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.012791 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.012774 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-envoy\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.012912 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.012893 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20c163a5-bae4-490f-9629-b550a1458f1f-istio-podinfo\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.018193 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.018167 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-istio-token\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.018321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.018303 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kdh8\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-kube-api-access-5kdh8\") pod \"e2e-ait-12b475ae-openshift-default-595458d449-t6gxk\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.140062 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.139965 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:12.278543 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.278520 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk"] Jul 06 18:10:12.280968 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:10:12.280940 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20c163a5_bae4_490f_9629_b550a1458f1f.slice/crio-fd9143f5365424b25a391b6064a2c47da8323cd9004f7b0c75e5936ea1c2fb4d WatchSource:0}: Error finding container fd9143f5365424b25a391b6064a2c47da8323cd9004f7b0c75e5936ea1c2fb4d: Status 404 returned error can't find the container with id fd9143f5365424b25a391b6064a2c47da8323cd9004f7b0c75e5936ea1c2fb4d Jul 06 18:10:12.283144 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.283107 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:10:12.283202 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.283182 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:10:12.283238 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.283210 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:10:12.728667 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.728632 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" event={"ID":"20c163a5-bae4-490f-9629-b550a1458f1f","Type":"ContainerStarted","Data":"bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2"} Jul 06 18:10:12.728864 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.728692 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" event={"ID":"20c163a5-bae4-490f-9629-b550a1458f1f","Type":"ContainerStarted","Data":"fd9143f5365424b25a391b6064a2c47da8323cd9004f7b0c75e5936ea1c2fb4d"} Jul 06 18:10:12.747131 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:12.747063 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" podStartSLOduration=1.747043944 podStartE2EDuration="1.747043944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:10:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:10:12.744753031 +0000 UTC m=+1780.150285959" watchObservedRunningTime="2026-07-06 18:10:12.747043944 +0000 UTC m=+1780.152576870" Jul 06 18:10:13.142007 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:13.141974 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:13.142620 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:13.142228 2558 patch_prober.go:28] interesting pod/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 06 18:10:13.142708 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:13.142615 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" podUID="20c163a5-bae4-490f-9629-b550a1458f1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 06 18:10:14.144183 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:14.144149 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:14.738165 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:14.738125 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:14.739159 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:14.739140 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:18.302214 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:18.302179 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk"] Jul 06 18:10:18.302672 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:18.302372 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" podUID="20c163a5-bae4-490f-9629-b550a1458f1f" containerName="istio-proxy" containerID="cri-o://bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2" gracePeriod=30 Jul 06 18:10:24.050877 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.050852 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:24.130210 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130127 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-credential-socket\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130210 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130164 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20c163a5-bae4-490f-9629-b550a1458f1f-istiod-ca-cert\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130210 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130198 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20c163a5-bae4-490f-9629-b550a1458f1f-istio-podinfo\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130226 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-istio-token\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130247 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-data\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130309 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-envoy\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130348 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kdh8\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-kube-api-access-5kdh8\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130417 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-socket\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130474 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-certs\") pod \"20c163a5-bae4-490f-9629-b550a1458f1f\" (UID: \"20c163a5-bae4-490f-9629-b550a1458f1f\") " Jul 06 18:10:24.130895 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130468 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:24.130895 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130609 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20c163a5-bae4-490f-9629-b550a1458f1f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:10:24.130895 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130730 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:24.130895 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.130862 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:24.131448 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.131391 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.131448 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.131399 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-data" (OuterVolumeSpecName: "istio-data") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:24.131448 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.131414 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.131448 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.131449 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.131723 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.131463 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20c163a5-bae4-490f-9629-b550a1458f1f-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.132718 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.132691 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20c163a5-bae4-490f-9629-b550a1458f1f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:10:24.132830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.132786 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-istio-token" (OuterVolumeSpecName: "istio-token") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:10:24.132963 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.132942 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-kube-api-access-5kdh8" (OuterVolumeSpecName: "kube-api-access-5kdh8") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "kube-api-access-5kdh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:10:24.133024 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.132943 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20c163a5-bae4-490f-9629-b550a1458f1f" (UID: "20c163a5-bae4-490f-9629-b550a1458f1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:24.232930 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.232896 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.232930 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.232923 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20c163a5-bae4-490f-9629-b550a1458f1f-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.232930 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.232932 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kdh8\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-kube-api-access-5kdh8\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.232930 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.232941 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20c163a5-bae4-490f-9629-b550a1458f1f-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.233173 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.232950 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20c163a5-bae4-490f-9629-b550a1458f1f-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:24.423785 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.423252 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8"] Jul 06 18:10:24.423941 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.423794 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c163a5-bae4-490f-9629-b550a1458f1f" containerName="istio-proxy" Jul 06 18:10:24.423941 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.423815 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c163a5-bae4-490f-9629-b550a1458f1f" containerName="istio-proxy" Jul 06 18:10:24.423941 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.423923 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="20c163a5-bae4-490f-9629-b550a1458f1f" containerName="istio-proxy" Jul 06 18:10:24.427222 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.427197 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.429714 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.429690 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5b603d23-openshift-default-dockercfg-dk6rd\"" Jul 06 18:10:24.445284 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.445244 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8"] Jul 06 18:10:24.536034 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.535999 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-envoy\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.536034 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536053 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxtch\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-kube-api-access-pxtch\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.537045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536094 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-socket\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.537045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536167 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-podinfo\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.537045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536259 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-data\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.537045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536289 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-credential-socket\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.537045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536312 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istiod-ca-cert\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.537045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536385 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-certs\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.537045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.536455 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-token\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.637891 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.637840 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-envoy\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.637899 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtch\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-kube-api-access-pxtch\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.637961 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-socket\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638000 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-podinfo\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638035 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-data\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638061 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-credential-socket\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638083 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istiod-ca-cert\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638414 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638130 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-certs\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638414 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638152 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-token\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638561 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638496 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-socket\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638561 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638537 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-data\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638736 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638714 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-certs\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.638822 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.638801 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-credential-socket\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.639051 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.639030 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istiod-ca-cert\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.640384 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.640363 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-envoy\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.640493 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.640467 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-podinfo\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.645598 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.645578 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-token\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.645846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.645827 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxtch\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-kube-api-access-pxtch\") pod \"e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.738893 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.738799 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:24.791698 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.791661 2558 generic.go:356] "Generic (PLEG): container finished" podID="20c163a5-bae4-490f-9629-b550a1458f1f" containerID="bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2" exitCode=0 Jul 06 18:10:24.791821 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.791741 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" Jul 06 18:10:24.791821 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.791741 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" event={"ID":"20c163a5-bae4-490f-9629-b550a1458f1f","Type":"ContainerDied","Data":"bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2"} Jul 06 18:10:24.791936 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.791846 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk" event={"ID":"20c163a5-bae4-490f-9629-b550a1458f1f","Type":"ContainerDied","Data":"fd9143f5365424b25a391b6064a2c47da8323cd9004f7b0c75e5936ea1c2fb4d"} Jul 06 18:10:24.791936 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.791884 2558 scope.go:117] "RemoveContainer" containerID="bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2" Jul 06 18:10:24.807446 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.807114 2558 scope.go:117] "RemoveContainer" containerID="bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2" Jul 06 18:10:24.808729 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:10:24.808648 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2\": container with ID starting with bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2 not found: ID does not exist" containerID="bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2" Jul 06 18:10:24.808729 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.808688 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2"} err="failed to get container status \"bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2\": rpc error: code = NotFound desc = could not find container \"bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2\": container with ID starting with bafdcdd6b39a75ed416b2043d5f5e1c31cd77e2662d80654287f0485d16d01b2 not found: ID does not exist" Jul 06 18:10:24.816054 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.816029 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk"] Jul 06 18:10:24.821458 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.821414 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-12b475ae-openshift-default-595458d449-t6gxk"] Jul 06 18:10:24.876321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.876295 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8"] Jul 06 18:10:24.878403 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:10:24.878376 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce3c492_3f46_4dab_b589_ea59e63e0b90.slice/crio-aa9bbbeb6c5c5efdbc4cdd72be86be3b4ad7c35a77beb7ddb946aa78e69102e5 WatchSource:0}: Error finding container aa9bbbeb6c5c5efdbc4cdd72be86be3b4ad7c35a77beb7ddb946aa78e69102e5: Status 404 returned error can't find the container with id aa9bbbeb6c5c5efdbc4cdd72be86be3b4ad7c35a77beb7ddb946aa78e69102e5 Jul 06 18:10:24.880438 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.880398 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:10:24.880529 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.880474 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:10:24.880529 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:24.880506 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:10:25.071982 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:25.071946 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20c163a5-bae4-490f-9629-b550a1458f1f" path="/var/lib/kubelet/pods/20c163a5-bae4-490f-9629-b550a1458f1f/volumes" Jul 06 18:10:25.797606 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:25.797569 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" event={"ID":"9ce3c492-3f46-4dab-b589-ea59e63e0b90","Type":"ContainerStarted","Data":"ce6702622320e1207e195884a5907cf054a5ec981b893a5db44ffc5061286b9b"} Jul 06 18:10:25.797830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:25.797627 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" event={"ID":"9ce3c492-3f46-4dab-b589-ea59e63e0b90","Type":"ContainerStarted","Data":"aa9bbbeb6c5c5efdbc4cdd72be86be3b4ad7c35a77beb7ddb946aa78e69102e5"} Jul 06 18:10:25.818875 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:25.818823 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podStartSLOduration=1.818805651 podStartE2EDuration="1.818805651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:10:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:10:25.817261383 +0000 UTC m=+1793.222794335" watchObservedRunningTime="2026-07-06 18:10:25.818805651 +0000 UTC m=+1793.224338576" Jul 06 18:10:26.740034 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:26.739998 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:26.741335 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:26.741311 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:26.741392 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:26.741362 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:27.740160 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:27.740121 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:27.740582 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:27.740194 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:28.739267 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:28.739229 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:28.739485 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:28.739288 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:29.740125 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:29.740087 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:29.740521 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:29.740143 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:30.739360 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:30.739320 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:30.739577 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:30.739390 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:31.739395 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:31.739358 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:31.739803 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:31.739459 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:32.740077 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:32.740038 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:32.740483 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:32.740100 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:33.070995 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:33.070964 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:10:33.072542 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:33.072521 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:10:33.075968 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:33.075949 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:10:33.077703 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:33.077687 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:10:33.739583 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:33.739546 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:33.739765 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:33.739601 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:34.739351 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:34.739314 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:34.739865 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:34.739374 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:35.740143 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:35.740106 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:35.740615 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:35.740197 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:35.847490 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:35.847389 2558 generic.go:356] "Generic (PLEG): container finished" podID="4f870b77-c67f-4d34-a43a-85d219f60110" containerID="aa69b805f16086832415cce7b9a7e0d55e96a493c24ab5112ed5942ad0815568" exitCode=1 Jul 06 18:10:35.847490 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:35.847462 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" event={"ID":"4f870b77-c67f-4d34-a43a-85d219f60110","Type":"ContainerDied","Data":"aa69b805f16086832415cce7b9a7e0d55e96a493c24ab5112ed5942ad0815568"} Jul 06 18:10:35.847849 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:35.847835 2558 scope.go:117] "RemoveContainer" containerID="aa69b805f16086832415cce7b9a7e0d55e96a493c24ab5112ed5942ad0815568" Jul 06 18:10:35.848152 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:35.848139 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 18:10:36.739935 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:36.739847 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:36.739935 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:36.739913 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:36.854756 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:36.854725 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" event={"ID":"4f870b77-c67f-4d34-a43a-85d219f60110","Type":"ContainerStarted","Data":"a1c5bfcfbeec3a7133e9731ae8ad11d3cf21a24edb2165176d396f69d646c21c"} Jul 06 18:10:36.855140 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:36.855039 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 18:10:37.740240 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:37.740200 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:37.740457 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:37.740259 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:38.739943 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:38.739906 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:38.740321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:38.739961 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:39.739393 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:39.739362 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:39.739606 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:39.739584 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:39.739658 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:39.739642 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:40.739385 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:40.739342 2558 patch_prober.go:28] interesting pod/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 06 18:10:40.739898 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:40.739419 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 06 18:10:41.660764 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:41.660728 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8"] Jul 06 18:10:41.660972 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:41.660946 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" containerID="cri-o://ce6702622320e1207e195884a5907cf054a5ec981b893a5db44ffc5061286b9b" gracePeriod=30 Jul 06 18:10:46.910683 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:46.910655 2558 generic.go:356] "Generic (PLEG): container finished" podID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerID="ce6702622320e1207e195884a5907cf054a5ec981b893a5db44ffc5061286b9b" exitCode=0 Jul 06 18:10:46.911033 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:46.910732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" event={"ID":"9ce3c492-3f46-4dab-b589-ea59e63e0b90","Type":"ContainerDied","Data":"ce6702622320e1207e195884a5907cf054a5ec981b893a5db44ffc5061286b9b"} Jul 06 18:10:46.911033 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:46.910780 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" event={"ID":"9ce3c492-3f46-4dab-b589-ea59e63e0b90","Type":"ContainerDied","Data":"aa9bbbeb6c5c5efdbc4cdd72be86be3b4ad7c35a77beb7ddb946aa78e69102e5"} Jul 06 18:10:46.911033 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:46.910802 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa9bbbeb6c5c5efdbc4cdd72be86be3b4ad7c35a77beb7ddb946aa78e69102e5" Jul 06 18:10:46.919695 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:46.919679 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:47.058591 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058550 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-data\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.058591 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058594 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-credential-socket\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.058857 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058621 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-token\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.058857 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058673 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-socket\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.058857 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058706 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-certs\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.059010 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058858 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-podinfo\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.059010 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058907 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-envoy\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.059010 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058893 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:47.059010 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.058910 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:47.059221 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059016 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istiod-ca-cert\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.059221 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059084 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxtch\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-kube-api-access-pxtch\") pod \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\" (UID: \"9ce3c492-3f46-4dab-b589-ea59e63e0b90\") " Jul 06 18:10:47.059221 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059016 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:47.059221 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059085 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:47.059452 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059296 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:10:47.059506 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059451 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.059506 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059467 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.059506 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059486 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.059506 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059504 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.059653 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.059518 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.061067 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.061040 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:10:47.061128 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.061081 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-kube-api-access-pxtch" (OuterVolumeSpecName: "kube-api-access-pxtch") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "kube-api-access-pxtch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:10:47.061243 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.061216 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:10:47.061350 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.061335 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ce3c492-3f46-4dab-b589-ea59e63e0b90" (UID: "9ce3c492-3f46-4dab-b589-ea59e63e0b90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:10:47.160631 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.160595 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxtch\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-kube-api-access-pxtch\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.160631 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.160622 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.160631 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.160631 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.160631 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.160640 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ce3c492-3f46-4dab-b589-ea59e63e0b90-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:10:47.862364 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.862332 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-5hrqw" Jul 06 18:10:47.916213 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.916184 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8" Jul 06 18:10:47.932234 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.932209 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8"] Jul 06 18:10:47.940213 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:47.940189 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5b603d23-openshift-default-566758c5c-nzmb8"] Jul 06 18:10:49.071496 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:10:49.071461 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" path="/var/lib/kubelet/pods/9ce3c492-3f46-4dab-b589-ea59e63e0b90/volumes" Jul 06 18:12:43.938390 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:43.938355 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g"] Jul 06 18:12:43.938913 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:43.938699 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" Jul 06 18:12:43.938913 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:43.938711 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" Jul 06 18:12:43.938913 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:43.938780 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce3c492-3f46-4dab-b589-ea59e63e0b90" containerName="istio-proxy" Jul 06 18:12:43.941768 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:43.941746 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:43.943809 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:43.943782 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-86957ffb-openshift-default-dockercfg-z5brj\"" Jul 06 18:12:43.960134 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:43.960111 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g"] Jul 06 18:12:44.055567 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.055529 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istiod-ca-cert\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.055829 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.055790 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-certs\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.055938 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.055868 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-data\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.055938 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.055908 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-credential-socket\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.056036 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.056001 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-envoy\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.056089 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.056048 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-socket\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.056145 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.056087 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-podinfo\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.056145 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.056112 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh4vt\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-kube-api-access-mh4vt\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.056250 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.056173 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-token\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.156943 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.156904 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istiod-ca-cert\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157146 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.156953 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-certs\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157146 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.156975 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-data\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157146 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-credential-socket\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157318 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157154 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-envoy\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157394 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-socket\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157401 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-data\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-podinfo\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157397 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-certs\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157533 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh4vt\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-kube-api-access-mh4vt\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157566 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-credential-socket\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157699 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-socket\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.157830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157702 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-token\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.158217 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.157843 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istiod-ca-cert\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.159481 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.159461 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-envoy\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.159837 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.159817 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-podinfo\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.164626 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.164605 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-token\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.164755 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.164738 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh4vt\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-kube-api-access-mh4vt\") pod \"e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.252152 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.252119 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:44.384905 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.384880 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g"] Jul 06 18:12:44.386634 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:12:44.386605 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d661f9_2ad9_4ff7_a7e8_dfd70bcf08f3.slice/crio-d2378e1d8726164e6a7ef5502db411b21bf47223570218356beb24632f78589a WatchSource:0}: Error finding container d2378e1d8726164e6a7ef5502db411b21bf47223570218356beb24632f78589a: Status 404 returned error can't find the container with id d2378e1d8726164e6a7ef5502db411b21bf47223570218356beb24632f78589a Jul 06 18:12:44.388953 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.388924 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:12:44.389029 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.388984 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:12:44.389029 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.389010 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:12:44.446162 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:44.446139 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" event={"ID":"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3","Type":"ContainerStarted","Data":"d2378e1d8726164e6a7ef5502db411b21bf47223570218356beb24632f78589a"} Jul 06 18:12:45.458907 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:45.458867 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" event={"ID":"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3","Type":"ContainerStarted","Data":"7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c"} Jul 06 18:12:45.478824 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:45.478773 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" podStartSLOduration=2.478758285 podStartE2EDuration="2.478758285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:12:45.476850273 +0000 UTC m=+1932.882383201" watchObservedRunningTime="2026-07-06 18:12:45.478758285 +0000 UTC m=+1932.884291211" Jul 06 18:12:46.252650 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:46.252607 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:46.258030 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:46.257999 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:46.463490 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:46.463464 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:12:46.464520 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:12:46.464498 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:13:02.505577 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:02.505536 2558 patch_prober.go:28] interesting pod/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 18:13:02.506023 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:02.505602 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 18:13:03.545530 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:03.545497 2558 generic.go:356] "Generic (PLEG): container finished" podID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerID="7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c" exitCode=0 Jul 06 18:13:03.545943 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:03.545569 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" event={"ID":"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3","Type":"ContainerDied","Data":"7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c"} Jul 06 18:13:03.546003 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:03.545987 2558 scope.go:117] "RemoveContainer" containerID="7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c" Jul 06 18:13:03.546903 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:03.546879 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:03.546953 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:03.546938 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:03.546987 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:03.546968 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:04.552502 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:04.552461 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" event={"ID":"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3","Type":"ContainerStarted","Data":"3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f"} Jul 06 18:13:05.253282 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:05.253251 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:13:05.256545 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:05.256523 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:13:05.458772 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:05.458741 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g"] Jul 06 18:13:05.558649 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:05.558558 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:13:05.558649 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:05.558556 2558 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" secret="" err="secret \"e2e-mt-86957ffb-openshift-default-dockercfg-z5brj\" not found" Jul 06 18:13:05.559578 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:05.559550 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:13:06.563330 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:06.563288 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" containerID="cri-o://3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f" gracePeriod=30 Jul 06 18:13:11.823927 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.823867 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:13:11.916469 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916445 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-credential-socket\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916474 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istiod-ca-cert\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916594 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916522 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-token\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916670 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916648 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-envoy\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916704 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916677 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mh4vt\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-kube-api-access-mh4vt\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916772 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916694 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:13:11.916836 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916738 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-data\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916836 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916828 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-certs\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916939 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916856 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-socket\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.916939 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.916895 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-podinfo\") pod \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\" (UID: \"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3\") " Jul 06 18:13:11.917132 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.917110 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:13:11.917176 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.917144 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:13:11.917292 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.917268 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:13:11.917359 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.917294 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:11.917359 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.917314 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:11.917359 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.917329 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:11.917634 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.917597 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:13:11.918891 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.918861 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:13:11.919004 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.918958 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:13:11.919063 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.918996 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:13:11.919063 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:11.919003 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-kube-api-access-mh4vt" (OuterVolumeSpecName: "kube-api-access-mh4vt") pod "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" (UID: "a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3"). InnerVolumeSpecName "kube-api-access-mh4vt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:13:12.018071 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.018039 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:12.018071 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.018066 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:12.018071 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.018078 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:12.018278 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.018090 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:12.018278 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.018101 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:12.018278 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.018111 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh4vt\" (UniqueName: \"kubernetes.io/projected/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3-kube-api-access-mh4vt\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:13:12.591953 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.591922 2558 generic.go:356] "Generic (PLEG): container finished" podID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerID="3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f" exitCode=0 Jul 06 18:13:12.592137 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.591970 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" Jul 06 18:13:12.592137 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.592013 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" event={"ID":"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3","Type":"ContainerDied","Data":"3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f"} Jul 06 18:13:12.592137 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.592075 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g" event={"ID":"a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3","Type":"ContainerDied","Data":"d2378e1d8726164e6a7ef5502db411b21bf47223570218356beb24632f78589a"} Jul 06 18:13:12.592137 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.592105 2558 scope.go:117] "RemoveContainer" containerID="3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f" Jul 06 18:13:12.601029 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.601013 2558 scope.go:117] "RemoveContainer" containerID="7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c" Jul 06 18:13:12.608222 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.608206 2558 scope.go:117] "RemoveContainer" containerID="3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f" Jul 06 18:13:12.608470 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:13:12.608454 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f\": container with ID starting with 3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f not found: ID does not exist" containerID="3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f" Jul 06 18:13:12.608528 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.608477 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f"} err="failed to get container status \"3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f\": rpc error: code = NotFound desc = could not find container \"3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f\": container with ID starting with 3c5c68aaffbac3b2af63af197408e28c60384bf7dc71072474f438935293172f not found: ID does not exist" Jul 06 18:13:12.608528 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.608491 2558 scope.go:117] "RemoveContainer" containerID="7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c" Jul 06 18:13:12.608717 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:13:12.608700 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c\": container with ID starting with 7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c not found: ID does not exist" containerID="7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c" Jul 06 18:13:12.608752 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.608723 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c"} err="failed to get container status \"7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c\": rpc error: code = NotFound desc = could not find container \"7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c\": container with ID starting with 7b6f4f6aba361f38b2326b5d2ecac289a3791802f2d0dd651439401f7333f80c not found: ID does not exist" Jul 06 18:13:12.612053 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.612034 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g"] Jul 06 18:13:12.616060 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:12.616041 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-86957ffb-openshift-default-749c59f9b7-xkq5g"] Jul 06 18:13:13.070659 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:13.070628 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" path="/var/lib/kubelet/pods/a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3/volumes" Jul 06 18:13:25.312363 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.312278 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q"] Jul 06 18:13:25.312945 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.312796 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" Jul 06 18:13:25.312945 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.312816 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" Jul 06 18:13:25.312945 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.312836 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" Jul 06 18:13:25.312945 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.312844 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" Jul 06 18:13:25.312945 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.312941 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" Jul 06 18:13:25.313200 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.312957 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d661f9-2ad9-4ff7-a7e8-dfd70bcf08f3" containerName="istio-proxy" Jul 06 18:13:25.317142 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.317122 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.319002 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.318985 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-14d0a8da-openshift-default-dockercfg-lg7qq\"" Jul 06 18:13:25.325725 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.325701 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q"] Jul 06 18:13:25.433737 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.433706 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkgvr\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-kube-api-access-dkgvr\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.433920 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.433760 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-envoy\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.433920 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.433815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-certs\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.433920 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.433834 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-socket\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.433920 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.433886 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-credential-socket\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.434091 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.433983 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-token\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.434091 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.434016 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-podinfo\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.434091 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.434049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-data\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.434091 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.434064 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee0d57d-238f-4da0-8f99-26f4950941ae-istiod-ca-cert\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535053 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535023 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-envoy\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-certs\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535088 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-socket\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535105 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-credential-socket\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535137 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-token\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-podinfo\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535418 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535191 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-data\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535418 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535213 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee0d57d-238f-4da0-8f99-26f4950941ae-istiod-ca-cert\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535418 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535249 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkgvr\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-kube-api-access-dkgvr\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535609 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535528 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-credential-socket\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535609 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535548 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-certs\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535728 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535702 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-socket\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.535935 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535916 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-data\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.536001 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.535949 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee0d57d-238f-4da0-8f99-26f4950941ae-istiod-ca-cert\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.537368 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.537351 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-envoy\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.537530 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.537511 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-podinfo\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.543106 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.543086 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-token\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.543461 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.543420 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkgvr\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-kube-api-access-dkgvr\") pod \"e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.627994 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.627943 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:25.763658 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.763628 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q"] Jul 06 18:13:25.766966 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:13:25.766929 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee0d57d_238f_4da0_8f99_26f4950941ae.slice/crio-5af7423a26dfdc87bdec26d98b833f29690d6d6b27541d50261de54528f95cb2 WatchSource:0}: Error finding container 5af7423a26dfdc87bdec26d98b833f29690d6d6b27541d50261de54528f95cb2: Status 404 returned error can't find the container with id 5af7423a26dfdc87bdec26d98b833f29690d6d6b27541d50261de54528f95cb2 Jul 06 18:13:25.769189 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.769145 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:25.769258 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.769239 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:25.769307 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:25.769292 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:26.671059 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:26.671025 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" event={"ID":"eee0d57d-238f-4da0-8f99-26f4950941ae","Type":"ContainerStarted","Data":"f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392"} Jul 06 18:13:26.671537 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:26.671080 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" event={"ID":"eee0d57d-238f-4da0-8f99-26f4950941ae","Type":"ContainerStarted","Data":"5af7423a26dfdc87bdec26d98b833f29690d6d6b27541d50261de54528f95cb2"} Jul 06 18:13:26.689889 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:26.689838 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" podStartSLOduration=1.689822683 podStartE2EDuration="1.689822683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:13:26.687952385 +0000 UTC m=+1974.093485312" watchObservedRunningTime="2026-07-06 18:13:26.689822683 +0000 UTC m=+1974.095355609" Jul 06 18:13:27.628693 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:27.628660 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:27.633313 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:27.633290 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:27.675665 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:27.675636 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:27.676509 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:27.676491 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:42.677224 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.677151 2558 patch_prober.go:28] interesting pod/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 06 18:13:42.677224 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.677205 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 06 18:13:42.745944 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.745914 2558 generic.go:356] "Generic (PLEG): container finished" podID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerID="f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392" exitCode=0 Jul 06 18:13:42.746080 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.745984 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" event={"ID":"eee0d57d-238f-4da0-8f99-26f4950941ae","Type":"ContainerDied","Data":"f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392"} Jul 06 18:13:42.746346 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.746331 2558 scope.go:117] "RemoveContainer" containerID="f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392" Jul 06 18:13:42.747278 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.747254 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:42.747335 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.747326 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:42.747403 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:42.747359 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:13:43.756149 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:43.756113 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" event={"ID":"eee0d57d-238f-4da0-8f99-26f4950941ae","Type":"ContainerStarted","Data":"ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4"} Jul 06 18:13:44.629082 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:44.629053 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:44.629315 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:44.629296 2558 patch_prober.go:28] interesting pod/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 06 18:13:44.629368 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:44.629352 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 06 18:13:45.629194 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:45.629160 2558 patch_prober.go:28] interesting pod/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 06 18:13:45.629630 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:45.629226 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 06 18:13:46.628313 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:46.628276 2558 patch_prober.go:28] interesting pod/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 06 18:13:46.628514 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:46.628334 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 06 18:13:47.632069 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:47.632039 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:47.632416 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:47.632376 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:13:47.633116 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:13:47.633100 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:15:06.863133 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:06.863055 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q"] Jul 06 18:15:06.863574 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:06.863299 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" containerID="cri-o://ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4" gracePeriod=30 Jul 06 18:15:12.104321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.104295 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:15:12.157760 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.157693 2558 generic.go:356] "Generic (PLEG): container finished" podID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerID="ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4" exitCode=0 Jul 06 18:15:12.157760 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.157733 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" Jul 06 18:15:12.157760 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.157743 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" event={"ID":"eee0d57d-238f-4da0-8f99-26f4950941ae","Type":"ContainerDied","Data":"ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4"} Jul 06 18:15:12.157988 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.157784 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q" event={"ID":"eee0d57d-238f-4da0-8f99-26f4950941ae","Type":"ContainerDied","Data":"5af7423a26dfdc87bdec26d98b833f29690d6d6b27541d50261de54528f95cb2"} Jul 06 18:15:12.157988 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.157805 2558 scope.go:117] "RemoveContainer" containerID="ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4" Jul 06 18:15:12.165815 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.165799 2558 scope.go:117] "RemoveContainer" containerID="f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392" Jul 06 18:15:12.173141 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.173125 2558 scope.go:117] "RemoveContainer" containerID="ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4" Jul 06 18:15:12.173365 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:15:12.173348 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4\": container with ID starting with ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4 not found: ID does not exist" containerID="ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4" Jul 06 18:15:12.173408 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.173374 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4"} err="failed to get container status \"ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4\": rpc error: code = NotFound desc = could not find container \"ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4\": container with ID starting with ee11ed806792df48163172e1017e201bb8982803c1beb353c95ad37f7388c7a4 not found: ID does not exist" Jul 06 18:15:12.173408 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.173388 2558 scope.go:117] "RemoveContainer" containerID="f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392" Jul 06 18:15:12.173670 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:15:12.173653 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392\": container with ID starting with f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392 not found: ID does not exist" containerID="f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392" Jul 06 18:15:12.173710 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.173678 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392"} err="failed to get container status \"f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392\": rpc error: code = NotFound desc = could not find container \"f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392\": container with ID starting with f9b2c91c0b244b335477603ef4911e4178fe85934630c26180133697d52f2392 not found: ID does not exist" Jul 06 18:15:12.189079 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189060 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-envoy\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189153 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189108 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-data\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189153 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189145 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-socket\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189219 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189193 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-certs\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189219 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189209 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-token\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189318 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189233 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-podinfo\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189318 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189251 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee0d57d-238f-4da0-8f99-26f4950941ae-istiod-ca-cert\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189318 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189298 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkgvr\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-kube-api-access-dkgvr\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189458 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189321 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-credential-socket\") pod \"eee0d57d-238f-4da0-8f99-26f4950941ae\" (UID: \"eee0d57d-238f-4da0-8f99-26f4950941ae\") " Jul 06 18:15:12.189516 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189476 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:15:12.189720 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189699 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.189789 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189739 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eee0d57d-238f-4da0-8f99-26f4950941ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:15:12.189845 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189797 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:15:12.189970 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.189911 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:15:12.190357 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.190340 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:15:12.191462 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.191414 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:15:12.191560 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.191501 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:15:12.191560 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.191497 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:15:12.191725 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.191712 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-kube-api-access-dkgvr" (OuterVolumeSpecName: "kube-api-access-dkgvr") pod "eee0d57d-238f-4da0-8f99-26f4950941ae" (UID: "eee0d57d-238f-4da0-8f99-26f4950941ae"). InnerVolumeSpecName "kube-api-access-dkgvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:15:12.290955 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290924 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.290955 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290949 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.290955 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290958 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.291175 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290965 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.291175 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290974 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee0d57d-238f-4da0-8f99-26f4950941ae-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.291175 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290982 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkgvr\" (UniqueName: \"kubernetes.io/projected/eee0d57d-238f-4da0-8f99-26f4950941ae-kube-api-access-dkgvr\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.291175 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290992 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.291175 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.290999 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee0d57d-238f-4da0-8f99-26f4950941ae-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:15:12.478714 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.478690 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q"] Jul 06 18:15:12.481906 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:12.481885 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-14d0a8da-openshift-default-6d7df4b87-ckh8q"] Jul 06 18:15:13.071734 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:13.071703 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" path="/var/lib/kubelet/pods/eee0d57d-238f-4da0-8f99-26f4950941ae/volumes" Jul 06 18:15:33.099087 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:33.099057 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:15:33.102587 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:33.102567 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:15:33.103247 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:33.103228 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:15:33.107013 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:15:33.106996 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:16:13.153574 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.153545 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh"] Jul 06 18:16:13.154168 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.154072 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" Jul 06 18:16:13.154168 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.154092 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" Jul 06 18:16:13.154168 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.154103 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" Jul 06 18:16:13.154168 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.154113 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" Jul 06 18:16:13.154375 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.154197 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" Jul 06 18:16:13.154375 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.154209 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee0d57d-238f-4da0-8f99-26f4950941ae" containerName="istio-proxy" Jul 06 18:16:13.157246 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.157226 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.159125 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.159108 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-205c4c-openshift-default-dockercfg-chg99\"" Jul 06 18:16:13.167120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.167099 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh"] Jul 06 18:16:13.254251 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254215 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202664a0-db48-411e-bc8d-ee915715972f-istio-podinfo\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254251 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9j66\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-kube-api-access-z9j66\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202664a0-db48-411e-bc8d-ee915715972f-istiod-ca-cert\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254329 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-istio-token\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-data\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254483 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-credential-socket\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254673 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254517 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-certs\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254673 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254538 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-envoy\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.254673 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.254636 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-socket\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356040 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356003 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9j66\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-kube-api-access-z9j66\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356210 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356049 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202664a0-db48-411e-bc8d-ee915715972f-istiod-ca-cert\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356210 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-istio-token\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356210 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356189 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-data\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356369 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356278 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-credential-socket\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356369 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356308 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-certs\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356369 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356328 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-envoy\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356554 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356380 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-socket\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356554 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356422 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202664a0-db48-411e-bc8d-ee915715972f-istio-podinfo\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356689 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356577 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-data\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356724 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356684 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-certs\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356758 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356717 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-credential-socket\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356845 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356829 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202664a0-db48-411e-bc8d-ee915715972f-istiod-ca-cert\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.356887 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.356836 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-socket\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.358760 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.358738 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-envoy\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.359005 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.358989 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202664a0-db48-411e-bc8d-ee915715972f-istio-podinfo\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.362996 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.362971 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-istio-token\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.363120 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.363104 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9j66\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-kube-api-access-z9j66\") pod \"e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.469671 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.467602 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:13.610136 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.610113 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh"] Jul 06 18:16:13.612172 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:16:13.612128 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202664a0_db48_411e_bc8d_ee915715972f.slice/crio-2212a01cbd4e292eef4ede62c2346b01895ea6e8b96956715491436f60b829e8 WatchSource:0}: Error finding container 2212a01cbd4e292eef4ede62c2346b01895ea6e8b96956715491436f60b829e8: Status 404 returned error can't find the container with id 2212a01cbd4e292eef4ede62c2346b01895ea6e8b96956715491436f60b829e8 Jul 06 18:16:13.614726 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.614692 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:13.614825 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.614775 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:13.614825 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:13.614820 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:14.452479 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:14.452421 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" event={"ID":"202664a0-db48-411e-bc8d-ee915715972f","Type":"ContainerStarted","Data":"a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50"} Jul 06 18:16:14.452958 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:14.452528 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" event={"ID":"202664a0-db48-411e-bc8d-ee915715972f","Type":"ContainerStarted","Data":"2212a01cbd4e292eef4ede62c2346b01895ea6e8b96956715491436f60b829e8"} Jul 06 18:16:14.469774 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:14.469739 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:14.471607 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:14.471567 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" podStartSLOduration=1.471552719 podStartE2EDuration="1.471552719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:16:14.469628057 +0000 UTC m=+2141.875160979" watchObservedRunningTime="2026-07-06 18:16:14.471552719 +0000 UTC m=+2141.877085708" Jul 06 18:16:14.475072 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:14.475052 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:15.457410 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:15.457379 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:15.458201 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:15.458179 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:21.456525 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.456496 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6"] Jul 06 18:16:21.460263 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.460242 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.462264 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.462245 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c34ce0-openshift-default-dockercfg-ttd8m\"" Jul 06 18:16:21.471362 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.471341 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6"] Jul 06 18:16:21.529735 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-credential-socket\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.529881 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529757 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-certs\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.529881 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529781 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f120390-f1e6-4082-bddf-20013a81c5aa-istiod-ca-cert\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.529881 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529798 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-podinfo\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.529881 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529817 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-socket\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.529881 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529854 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-data\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.530041 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529885 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-token\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.530041 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529905 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-envoy\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.530041 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.529927 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdtlj\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-kube-api-access-tdtlj\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630748 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630720 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f120390-f1e6-4082-bddf-20013a81c5aa-istiod-ca-cert\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630748 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630753 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-podinfo\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630985 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630775 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-socket\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630985 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-data\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630985 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630829 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-token\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630985 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630848 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-envoy\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630985 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630862 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdtlj\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-kube-api-access-tdtlj\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.630985 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630911 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-credential-socket\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.631291 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.630993 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-certs\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.631291 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.631217 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-data\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.631454 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.631410 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-socket\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.631532 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.631450 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-certs\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.631532 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.631474 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-credential-socket\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.631640 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.631594 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f120390-f1e6-4082-bddf-20013a81c5aa-istiod-ca-cert\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.633173 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.633148 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-envoy\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.633318 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.633304 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-podinfo\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.637458 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.637421 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-token\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.637712 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.637692 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdtlj\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-kube-api-access-tdtlj\") pod \"e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.771639 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.771610 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:21.902988 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.902967 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6"] Jul 06 18:16:21.905169 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:16:21.905134 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f120390_f1e6_4082_bddf_20013a81c5aa.slice/crio-081774b4d598c42410b5229e8130c21148084242c570707cc69ffa814ebbafaa WatchSource:0}: Error finding container 081774b4d598c42410b5229e8130c21148084242c570707cc69ffa814ebbafaa: Status 404 returned error can't find the container with id 081774b4d598c42410b5229e8130c21148084242c570707cc69ffa814ebbafaa Jul 06 18:16:21.907457 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.907404 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:21.907528 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.907493 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:21.907528 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:21.907523 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:22.494374 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:22.494331 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" event={"ID":"3f120390-f1e6-4082-bddf-20013a81c5aa","Type":"ContainerStarted","Data":"7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8"} Jul 06 18:16:22.494846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:22.494394 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" event={"ID":"3f120390-f1e6-4082-bddf-20013a81c5aa","Type":"ContainerStarted","Data":"081774b4d598c42410b5229e8130c21148084242c570707cc69ffa814ebbafaa"} Jul 06 18:16:22.515193 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:22.515141 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podStartSLOduration=1.515126667 podStartE2EDuration="1.515126667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:16:22.511484257 +0000 UTC m=+2149.917017183" watchObservedRunningTime="2026-07-06 18:16:22.515126667 +0000 UTC m=+2149.920659650" Jul 06 18:16:22.772475 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:22.772444 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:22.777114 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:22.777091 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:23.499161 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:23.499129 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:23.500240 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:23.500220 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:31.499637 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:31.499597 2558 patch_prober.go:28] interesting pod/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 18:16:31.500053 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:31.499661 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 18:16:32.548203 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:32.548159 2558 generic.go:356] "Generic (PLEG): container finished" podID="202664a0-db48-411e-bc8d-ee915715972f" containerID="a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50" exitCode=0 Jul 06 18:16:32.548761 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:32.548266 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" event={"ID":"202664a0-db48-411e-bc8d-ee915715972f","Type":"ContainerDied","Data":"a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50"} Jul 06 18:16:32.548843 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:32.548815 2558 scope.go:117] "RemoveContainer" containerID="a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50" Jul 06 18:16:32.550320 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:32.550281 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:32.550410 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:32.550383 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:32.550495 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:32.550450 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:33.126419 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:33.126388 2558 scope.go:117] "RemoveContainer" containerID="ce6702622320e1207e195884a5907cf054a5ec981b893a5db44ffc5061286b9b" Jul 06 18:16:33.557738 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:33.557695 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" event={"ID":"202664a0-db48-411e-bc8d-ee915715972f","Type":"ContainerStarted","Data":"dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e"} Jul 06 18:16:34.470463 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:34.470408 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:34.471150 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:34.471124 2558 patch_prober.go:28] interesting pod/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 06 18:16:34.471373 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:34.471339 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 06 18:16:35.470972 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:35.470933 2558 patch_prober.go:28] interesting pod/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 06 18:16:35.471479 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:35.471001 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 06 18:16:36.470481 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:36.470368 2558 patch_prober.go:28] interesting pod/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 06 18:16:36.470481 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:36.470448 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 06 18:16:37.473778 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:37.473740 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:37.474279 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:37.474129 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:37.474866 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:37.474846 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:16:38.999589 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:38.999558 2558 patch_prober.go:28] interesting pod/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": read tcp 10.134.0.2:60664->10.134.0.35:15021: read: connection reset by peer" start-of-body= Jul 06 18:16:39.000149 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:38.999640 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": read tcp 10.134.0.2:60664->10.134.0.35:15021: read: connection reset by peer" Jul 06 18:16:39.036667 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:16:39.036637 2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f120390_f1e6_4082_bddf_20013a81c5aa.slice/crio-conmon-7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8.scope\": RecentStats: unable to find data in memory cache]" Jul 06 18:16:39.594765 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:39.594732 2558 generic.go:356] "Generic (PLEG): container finished" podID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerID="7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8" exitCode=0 Jul 06 18:16:39.594929 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:39.594805 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" event={"ID":"3f120390-f1e6-4082-bddf-20013a81c5aa","Type":"ContainerDied","Data":"7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8"} Jul 06 18:16:39.595255 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:39.595240 2558 scope.go:117] "RemoveContainer" containerID="7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8" Jul 06 18:16:39.596246 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:39.596217 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:39.596418 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:39.596300 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:39.596418 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:39.596351 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:16:40.600862 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:40.600832 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" event={"ID":"3f120390-f1e6-4082-bddf-20013a81c5aa","Type":"ContainerStarted","Data":"8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75"} Jul 06 18:16:40.772302 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:40.772264 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:40.772516 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:40.772493 2558 patch_prober.go:28] interesting pod/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 06 18:16:40.772569 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:40.772541 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 06 18:16:41.772970 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:41.772931 2558 patch_prober.go:28] interesting pod/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 06 18:16:41.773461 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:41.772988 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 06 18:16:42.772072 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:42.772036 2558 patch_prober.go:28] interesting pod/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 06 18:16:42.772277 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:42.772097 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 06 18:16:43.772821 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:43.772784 2558 patch_prober.go:28] interesting pod/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 06 18:16:43.773254 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:43.772839 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 06 18:16:44.775394 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:44.775362 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:44.775900 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:44.775689 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:16:44.776412 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:16:44.776391 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:18:06.000250 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:06.000169 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh"] Jul 06 18:18:06.000714 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:06.000419 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" containerID="cri-o://dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e" gracePeriod=30 Jul 06 18:18:07.475879 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:07.475846 2558 patch_prober.go:28] interesting pod/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 18:18:07.476250 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:07.475928 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 18:18:11.253769 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.253740 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:18:11.314007 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.313936 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-socket\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314007 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.313973 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-certs\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314195 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314018 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202664a0-db48-411e-bc8d-ee915715972f-istiod-ca-cert\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314195 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314039 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-envoy\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314195 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314063 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202664a0-db48-411e-bc8d-ee915715972f-istio-podinfo\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314195 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314126 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-data\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314393 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314221 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:11.314393 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314319 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9j66\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-kube-api-access-z9j66\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314393 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314320 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:11.314393 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314358 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-credential-socket\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314393 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314385 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-istio-token\") pod \"202664a0-db48-411e-bc8d-ee915715972f\" (UID: \"202664a0-db48-411e-bc8d-ee915715972f\") " Jul 06 18:18:11.314695 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314470 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/202664a0-db48-411e-bc8d-ee915715972f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:18:11.314695 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314594 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:11.314821 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314800 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.314883 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314826 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.314883 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314840 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202664a0-db48-411e-bc8d-ee915715972f-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.314883 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.314850 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.315281 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.315260 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-data" (OuterVolumeSpecName: "istio-data") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:11.316474 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.316405 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:11.316576 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.316556 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-istio-token" (OuterVolumeSpecName: "istio-token") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:11.316734 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.316703 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/202664a0-db48-411e-bc8d-ee915715972f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:18:11.316803 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.316708 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-kube-api-access-z9j66" (OuterVolumeSpecName: "kube-api-access-z9j66") pod "202664a0-db48-411e-bc8d-ee915715972f" (UID: "202664a0-db48-411e-bc8d-ee915715972f"). InnerVolumeSpecName "kube-api-access-z9j66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:11.415604 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.415575 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.415604 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.415600 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9j66\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-kube-api-access-z9j66\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.415604 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.415611 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202664a0-db48-411e-bc8d-ee915715972f-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.415809 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.415621 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202664a0-db48-411e-bc8d-ee915715972f-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:11.415809 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:11.415628 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202664a0-db48-411e-bc8d-ee915715972f-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:12.028448 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.028401 2558 generic.go:356] "Generic (PLEG): container finished" podID="202664a0-db48-411e-bc8d-ee915715972f" containerID="dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e" exitCode=0 Jul 06 18:18:12.028648 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.028480 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" Jul 06 18:18:12.028648 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.028492 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" event={"ID":"202664a0-db48-411e-bc8d-ee915715972f","Type":"ContainerDied","Data":"dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e"} Jul 06 18:18:12.028648 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.028557 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh" event={"ID":"202664a0-db48-411e-bc8d-ee915715972f","Type":"ContainerDied","Data":"2212a01cbd4e292eef4ede62c2346b01895ea6e8b96956715491436f60b829e8"} Jul 06 18:18:12.028648 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.028586 2558 scope.go:117] "RemoveContainer" containerID="dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e" Jul 06 18:18:12.037983 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.037969 2558 scope.go:117] "RemoveContainer" containerID="a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50" Jul 06 18:18:12.045370 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.045346 2558 scope.go:117] "RemoveContainer" containerID="dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e" Jul 06 18:18:12.045774 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:12.045717 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e\": container with ID starting with dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e not found: ID does not exist" containerID="dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e" Jul 06 18:18:12.045931 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.045751 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e"} err="failed to get container status \"dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e\": rpc error: code = NotFound desc = could not find container \"dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e\": container with ID starting with dcefb427f3d2ae58c6f085cb1b1da711cef72ef2405ed16cccea79d6628a9e9e not found: ID does not exist" Jul 06 18:18:12.045931 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.045801 2558 scope.go:117] "RemoveContainer" containerID="a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50" Jul 06 18:18:12.046344 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:12.046327 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50\": container with ID starting with a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50 not found: ID does not exist" containerID="a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50" Jul 06 18:18:12.046446 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.046362 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50"} err="failed to get container status \"a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50\": rpc error: code = NotFound desc = could not find container \"a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50\": container with ID starting with a0a6553ab88b613a629c8559c325fe25fadb9fc18a1de304d048d70db8d40f50 not found: ID does not exist" Jul 06 18:18:12.047781 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.047761 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh"] Jul 06 18:18:12.050922 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.050903 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-205c4c-openshift-default-5bc6d775d7-5rgbh"] Jul 06 18:18:12.808300 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.808268 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6"] Jul 06 18:18:12.808779 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:12.808541 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" containerID="cri-o://8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75" gracePeriod=30 Jul 06 18:18:13.070259 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.070182 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="202664a0-db48-411e-bc8d-ee915715972f" path="/var/lib/kubelet/pods/202664a0-db48-411e-bc8d-ee915715972f/volumes" Jul 06 18:18:13.667372 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.667341 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s"] Jul 06 18:18:13.667775 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.667759 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" Jul 06 18:18:13.667869 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.667778 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" Jul 06 18:18:13.667869 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.667791 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" Jul 06 18:18:13.667869 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.667800 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" Jul 06 18:18:13.668012 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.667900 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" Jul 06 18:18:13.668012 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.667914 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="202664a0-db48-411e-bc8d-ee915715972f" containerName="istio-proxy" Jul 06 18:18:13.672329 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.672307 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.674261 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.674243 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-aa698f-openshift-default-dockercfg-4k6mv\"" Jul 06 18:18:13.680587 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.680565 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s"] Jul 06 18:18:13.740089 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740054 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-certs\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740274 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-credential-socket\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740274 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740127 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istiod-ca-cert\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740274 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-data\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740274 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740180 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-socket\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740513 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740279 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-podinfo\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740513 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740334 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-envoy\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740513 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740439 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd98m\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-kube-api-access-sd98m\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.740513 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.740475 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-token\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841194 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841161 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-certs\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841201 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-credential-socket\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841228 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istiod-ca-cert\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841261 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-data\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841295 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-socket\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841351 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-podinfo\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841382 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-envoy\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841484 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd98m\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-kube-api-access-sd98m\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.841636 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841504 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-token\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.842058 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841699 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-credential-socket\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.842058 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841757 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-data\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.842058 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841819 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-socket\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.842058 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.841904 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-certs\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.842058 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.842045 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istiod-ca-cert\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.843615 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.843594 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-envoy\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.843889 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.843868 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-podinfo\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.849554 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.849533 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd98m\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-kube-api-access-sd98m\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.849836 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.849811 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-token\") pod \"e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:13.983716 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:13.983625 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:14.120687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:14.120664 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s"] Jul 06 18:18:14.122156 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:18:14.122127 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7568ab_4b1e_49f8_96f4_6c192f2617d7.slice/crio-2d00baf3953f255021908c16b30b794f6ab905ad740b2b56abdb0f3f937abd7e WatchSource:0}: Error finding container 2d00baf3953f255021908c16b30b794f6ab905ad740b2b56abdb0f3f937abd7e: Status 404 returned error can't find the container with id 2d00baf3953f255021908c16b30b794f6ab905ad740b2b56abdb0f3f937abd7e Jul 06 18:18:14.124488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:14.124452 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:14.124573 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:14.124519 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:14.124573 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:14.124554 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:14.778667 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:14.778629 2558 patch_prober.go:28] interesting pod/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 18:18:14.778831 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:14.778688 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 18:18:15.047439 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:15.047344 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" event={"ID":"ed7568ab-4b1e-49f8-96f4-6c192f2617d7","Type":"ContainerStarted","Data":"40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6"} Jul 06 18:18:15.047800 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:15.047422 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" event={"ID":"ed7568ab-4b1e-49f8-96f4-6c192f2617d7","Type":"ContainerStarted","Data":"2d00baf3953f255021908c16b30b794f6ab905ad740b2b56abdb0f3f937abd7e"} Jul 06 18:18:15.067046 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:15.066997 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" podStartSLOduration=2.066983239 podStartE2EDuration="2.066983239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:15.065177023 +0000 UTC m=+2262.470709977" watchObservedRunningTime="2026-07-06 18:18:15.066983239 +0000 UTC m=+2262.472516163" Jul 06 18:18:15.984238 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:15.984207 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:15.988960 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:15.988937 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:16.052033 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:16.052001 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:16.052901 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:16.052882 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:18.050082 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.050061 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:18:18.062131 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.062106 2558 generic.go:356] "Generic (PLEG): container finished" podID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerID="8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75" exitCode=0 Jul 06 18:18:18.062256 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.062144 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" Jul 06 18:18:18.062256 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.062184 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" event={"ID":"3f120390-f1e6-4082-bddf-20013a81c5aa","Type":"ContainerDied","Data":"8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75"} Jul 06 18:18:18.062256 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.062230 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6" event={"ID":"3f120390-f1e6-4082-bddf-20013a81c5aa","Type":"ContainerDied","Data":"081774b4d598c42410b5229e8130c21148084242c570707cc69ffa814ebbafaa"} Jul 06 18:18:18.062256 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.062254 2558 scope.go:117] "RemoveContainer" containerID="8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75" Jul 06 18:18:18.070077 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.070050 2558 scope.go:117] "RemoveContainer" containerID="7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8" Jul 06 18:18:18.079102 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.079081 2558 scope.go:117] "RemoveContainer" containerID="8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75" Jul 06 18:18:18.079407 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:18.079381 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75\": container with ID starting with 8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75 not found: ID does not exist" containerID="8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75" Jul 06 18:18:18.079515 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.079413 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75"} err="failed to get container status \"8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75\": rpc error: code = NotFound desc = could not find container \"8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75\": container with ID starting with 8f6be08403b86c9c09ce8043fb448b5f1cd2ede9b87a2240d208325592d7fa75 not found: ID does not exist" Jul 06 18:18:18.079515 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.079455 2558 scope.go:117] "RemoveContainer" containerID="7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8" Jul 06 18:18:18.079740 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:18.079718 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8\": container with ID starting with 7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8 not found: ID does not exist" containerID="7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8" Jul 06 18:18:18.079806 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.079747 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8"} err="failed to get container status \"7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8\": rpc error: code = NotFound desc = could not find container \"7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8\": container with ID starting with 7fbe180eb80e2e4ef441cc4b23d9a522d8c0bcae8c83792f33691ccc90c90fa8 not found: ID does not exist" Jul 06 18:18:18.187365 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187334 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-podinfo\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187553 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187386 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f120390-f1e6-4082-bddf-20013a81c5aa-istiod-ca-cert\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187553 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187406 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-token\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187553 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187459 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-envoy\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187553 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187482 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdtlj\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-kube-api-access-tdtlj\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187553 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187517 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-data\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187801 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187560 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-certs\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187801 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187590 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-credential-socket\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187801 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187701 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-socket\") pod \"3f120390-f1e6-4082-bddf-20013a81c5aa\" (UID: \"3f120390-f1e6-4082-bddf-20013a81c5aa\") " Jul 06 18:18:18.187969 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187855 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:18.187969 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.187879 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f120390-f1e6-4082-bddf-20013a81c5aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:18:18.188067 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.188042 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:18.188193 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.188147 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:18.188270 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.188244 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f120390-f1e6-4082-bddf-20013a81c5aa-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.188270 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.188263 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.188379 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.188279 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.188379 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.188295 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.188379 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.188366 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:18.189731 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.189709 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:18.189840 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.189777 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-kube-api-access-tdtlj" (OuterVolumeSpecName: "kube-api-access-tdtlj") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "kube-api-access-tdtlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:18.189893 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.189878 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:18.190023 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.190009 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f120390-f1e6-4082-bddf-20013a81c5aa" (UID: "3f120390-f1e6-4082-bddf-20013a81c5aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:18:18.289379 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.289346 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.289379 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.289372 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.289379 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.289381 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.289607 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.289389 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdtlj\" (UniqueName: \"kubernetes.io/projected/3f120390-f1e6-4082-bddf-20013a81c5aa-kube-api-access-tdtlj\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.289607 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.289398 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f120390-f1e6-4082-bddf-20013a81c5aa-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:18.389477 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.389417 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6"] Jul 06 18:18:18.397333 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:18.397296 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c34ce0-openshift-default-789cd44c7f-pfxh6"] Jul 06 18:18:19.070691 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:19.070658 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" path="/var/lib/kubelet/pods/3f120390-f1e6-4082-bddf-20013a81c5aa/volumes" Jul 06 18:18:22.164448 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.164396 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5"] Jul 06 18:18:22.164803 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.164744 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" Jul 06 18:18:22.164803 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.164755 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" Jul 06 18:18:22.164876 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.164837 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" Jul 06 18:18:22.164876 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.164845 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" Jul 06 18:18:22.169299 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.169262 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.171357 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.171333 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-77c307-openshift-default-dockercfg-vm97f\"" Jul 06 18:18:22.178747 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.178727 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5"] Jul 06 18:18:22.226252 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226224 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-socket\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226420 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226268 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98cf782b-6ecf-4a7b-9b58-538430707391-istio-podinfo\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226420 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226289 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-istio-token\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226420 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226307 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-certs\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226420 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226383 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-envoy\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226612 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226481 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-data\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226612 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226515 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98cf782b-6ecf-4a7b-9b58-538430707391-istiod-ca-cert\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226612 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226580 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-credential-socket\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.226714 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.226630 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n927\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-kube-api-access-9n927\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328066 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328022 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-envoy\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328235 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328084 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-data\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328235 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328115 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98cf782b-6ecf-4a7b-9b58-538430707391-istiod-ca-cert\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328235 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328181 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-credential-socket\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328412 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328247 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n927\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-kube-api-access-9n927\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328412 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328294 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-socket\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328412 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98cf782b-6ecf-4a7b-9b58-538430707391-istio-podinfo\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328412 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328360 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-istio-token\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328412 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328392 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-certs\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.328793 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.328768 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-certs\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.329031 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.329008 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-data\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.329289 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.329267 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-credential-socket\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.329675 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.329650 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98cf782b-6ecf-4a7b-9b58-538430707391-istiod-ca-cert\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.329821 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.329795 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-socket\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.336620 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.331000 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-envoy\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.336620 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.331306 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98cf782b-6ecf-4a7b-9b58-538430707391-istio-podinfo\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.336620 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.336556 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-istio-token\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.336855 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.336711 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n927\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-kube-api-access-9n927\") pod \"e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.479495 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.479399 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:22.610590 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.610567 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5"] Jul 06 18:18:22.612533 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:18:22.612504 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98cf782b_6ecf_4a7b_9b58_538430707391.slice/crio-17ee3b4dfe4e3822693cb134432330da80dc3a07d368228e19149b0812ed064d WatchSource:0}: Error finding container 17ee3b4dfe4e3822693cb134432330da80dc3a07d368228e19149b0812ed064d: Status 404 returned error can't find the container with id 17ee3b4dfe4e3822693cb134432330da80dc3a07d368228e19149b0812ed064d Jul 06 18:18:22.614574 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.614544 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:22.614650 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.614605 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:22.614650 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:22.614636 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:23.086488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:23.086453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" event={"ID":"98cf782b-6ecf-4a7b-9b58-538430707391","Type":"ContainerStarted","Data":"d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa"} Jul 06 18:18:23.086644 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:23.086504 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" event={"ID":"98cf782b-6ecf-4a7b-9b58-538430707391","Type":"ContainerStarted","Data":"17ee3b4dfe4e3822693cb134432330da80dc3a07d368228e19149b0812ed064d"} Jul 06 18:18:23.108186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:23.108132 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" podStartSLOduration=1.108110508 podStartE2EDuration="1.108110508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:23.106062739 +0000 UTC m=+2270.511595665" watchObservedRunningTime="2026-07-06 18:18:23.108110508 +0000 UTC m=+2270.513643434" Jul 06 18:18:23.479639 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:23.479554 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:23.484304 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:23.484282 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:24.090832 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:24.090796 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:24.091599 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:24.091584 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:32.053456 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.052055 2558 patch_prober.go:28] interesting pod/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:40006->10.134.0.36:15021: read: connection reset by peer" start-of-body= Jul 06 18:18:32.053456 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.052144 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": read tcp 10.134.0.2:40006->10.134.0.36:15021: read: connection reset by peer" Jul 06 18:18:32.127547 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.127514 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerID="40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6" exitCode=0 Jul 06 18:18:32.127670 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.127588 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" event={"ID":"ed7568ab-4b1e-49f8-96f4-6c192f2617d7","Type":"ContainerDied","Data":"40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6"} Jul 06 18:18:32.128013 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.127997 2558 scope.go:117] "RemoveContainer" containerID="40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6" Jul 06 18:18:32.128982 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.128953 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:32.129038 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.129022 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:32.129075 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:32.129052 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:33.137547 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:33.137509 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" event={"ID":"ed7568ab-4b1e-49f8-96f4-6c192f2617d7","Type":"ContainerStarted","Data":"d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f"} Jul 06 18:18:33.984442 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:33.984395 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:33.984656 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:33.984634 2558 patch_prober.go:28] interesting pod/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 06 18:18:33.984707 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:33.984676 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 06 18:18:34.984543 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:34.984504 2558 patch_prober.go:28] interesting pod/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 06 18:18:34.984937 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:34.984568 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 06 18:18:35.984735 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:35.984694 2558 patch_prober.go:28] interesting pod/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 06 18:18:35.985210 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:35.984762 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 06 18:18:36.987582 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:36.987551 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:36.988035 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:36.987821 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:36.988405 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:36.988390 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:40.132561 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:40.132526 2558 patch_prober.go:28] interesting pod/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 18:18:40.133068 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:40.132606 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 18:18:41.183666 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:41.183634 2558 generic.go:356] "Generic (PLEG): container finished" podID="98cf782b-6ecf-4a7b-9b58-538430707391" containerID="d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa" exitCode=0 Jul 06 18:18:41.184079 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:41.183671 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" event={"ID":"98cf782b-6ecf-4a7b-9b58-538430707391","Type":"ContainerDied","Data":"d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa"} Jul 06 18:18:41.184175 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:41.184158 2558 scope.go:117] "RemoveContainer" containerID="d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa" Jul 06 18:18:41.185163 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:41.185137 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:41.185221 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:41.185199 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:41.185259 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:41.185230 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:42.189932 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:42.189898 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" event={"ID":"98cf782b-6ecf-4a7b-9b58-538430707391","Type":"ContainerStarted","Data":"837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567"} Jul 06 18:18:42.480494 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:42.480403 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:42.480668 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:42.480645 2558 patch_prober.go:28] interesting pod/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 06 18:18:42.480715 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:42.480695 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 06 18:18:43.480548 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:43.480511 2558 patch_prober.go:28] interesting pod/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 06 18:18:43.480938 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:43.480565 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 06 18:18:43.779181 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:43.779140 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s"] Jul 06 18:18:43.779492 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:43.779465 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" containerID="cri-o://d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f" gracePeriod=30 Jul 06 18:18:44.479992 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:44.479959 2558 patch_prober.go:28] interesting pod/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 06 18:18:44.480209 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:44.480020 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 06 18:18:45.480101 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:45.480063 2558 patch_prober.go:28] interesting pod/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 06 18:18:45.480492 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:45.480137 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 06 18:18:46.483888 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:46.483857 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:46.484252 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:46.484222 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:46.485019 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:46.484997 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:49.029506 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.029473 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:49.105689 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105616 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-certs\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105689 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105657 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-podinfo\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105689 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105688 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-data\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105713 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-envoy\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105733 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-token\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105756 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-socket\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105785 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-credential-socket\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105882 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istiod-ca-cert\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.105964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105892 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:49.105964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.105907 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd98m\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-kube-api-access-sd98m\") pod \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\" (UID: \"ed7568ab-4b1e-49f8-96f4-6c192f2617d7\") " Jul 06 18:18:49.106321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.106151 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:49.106321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.106165 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:49.106321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.106213 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.106321 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.106225 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:18:49.106536 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.106480 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:49.107961 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.107936 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:49.108133 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.108110 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:49.108486 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.108457 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-kube-api-access-sd98m" (OuterVolumeSpecName: "kube-api-access-sd98m") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "kube-api-access-sd98m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:49.108578 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.108486 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed7568ab-4b1e-49f8-96f4-6c192f2617d7" (UID: "ed7568ab-4b1e-49f8-96f4-6c192f2617d7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:18:49.207186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207149 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.207186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207182 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.207186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207191 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.207408 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207199 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.207408 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207207 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.207408 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207214 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.207408 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207223 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.207408 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.207233 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd98m\" (UniqueName: \"kubernetes.io/projected/ed7568ab-4b1e-49f8-96f4-6c192f2617d7-kube-api-access-sd98m\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:49.223648 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.223623 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerID="d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f" exitCode=0 Jul 06 18:18:49.223753 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.223682 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" Jul 06 18:18:49.223753 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.223702 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" event={"ID":"ed7568ab-4b1e-49f8-96f4-6c192f2617d7","Type":"ContainerDied","Data":"d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f"} Jul 06 18:18:49.223828 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.223755 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s" event={"ID":"ed7568ab-4b1e-49f8-96f4-6c192f2617d7","Type":"ContainerDied","Data":"2d00baf3953f255021908c16b30b794f6ab905ad740b2b56abdb0f3f937abd7e"} Jul 06 18:18:49.223828 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.223777 2558 scope.go:117] "RemoveContainer" containerID="d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f" Jul 06 18:18:49.232925 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.232910 2558 scope.go:117] "RemoveContainer" containerID="40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6" Jul 06 18:18:49.240972 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.240955 2558 scope.go:117] "RemoveContainer" containerID="d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f" Jul 06 18:18:49.241285 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:49.241263 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f\": container with ID starting with d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f not found: ID does not exist" containerID="d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f" Jul 06 18:18:49.241361 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.241292 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f"} err="failed to get container status \"d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f\": rpc error: code = NotFound desc = could not find container \"d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f\": container with ID starting with d930f93640afa5a44e3eb5fb2e56822f6c167c4838113d8264a2e0e3e4d2396f not found: ID does not exist" Jul 06 18:18:49.241361 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.241308 2558 scope.go:117] "RemoveContainer" containerID="40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6" Jul 06 18:18:49.241609 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:49.241592 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6\": container with ID starting with 40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6 not found: ID does not exist" containerID="40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6" Jul 06 18:18:49.241706 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.241615 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6"} err="failed to get container status \"40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6\": rpc error: code = NotFound desc = could not find container \"40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6\": container with ID starting with 40fb459528ca43d09eedf6fa72912f686c4691285e3c166a9ab4fe39b57f8bc6 not found: ID does not exist" Jul 06 18:18:49.243719 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.243702 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s"] Jul 06 18:18:49.246640 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:49.246618 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-aa698f-openshift-default-577757df4f-t7x4s"] Jul 06 18:18:50.528638 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:50.528605 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5"] Jul 06 18:18:50.529038 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:50.528834 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" containerID="cri-o://837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567" gracePeriod=30 Jul 06 18:18:51.070771 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.070737 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" path="/var/lib/kubelet/pods/ed7568ab-4b1e-49f8-96f4-6c192f2617d7/volumes" Jul 06 18:18:51.368675 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.368599 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd"] Jul 06 18:18:51.368957 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.368945 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" Jul 06 18:18:51.368999 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.368958 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" Jul 06 18:18:51.368999 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.368976 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" Jul 06 18:18:51.368999 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.368982 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f120390-f1e6-4082-bddf-20013a81c5aa" containerName="istio-proxy" Jul 06 18:18:51.369092 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.369037 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" Jul 06 18:18:51.369092 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.369048 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" Jul 06 18:18:51.371925 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.371906 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.373825 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.373803 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-91cb9f-openshift-default-dockercfg-mzzmn\"" Jul 06 18:18:51.383176 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.383155 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd"] Jul 06 18:18:51.429394 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429370 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-envoy\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429525 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-data\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429525 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-credential-socket\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429610 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429530 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-socket\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429610 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429601 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3c18502-fb64-4f99-bb93-1bafd11bb517-istiod-ca-cert\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429629 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-certs\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429647 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-token\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429760 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429692 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvnf8\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-kube-api-access-pvnf8\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.429760 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.429724 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-podinfo\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.530723 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530685 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-data\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.530723 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530726 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-credential-socket\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530746 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-socket\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530811 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3c18502-fb64-4f99-bb93-1bafd11bb517-istiod-ca-cert\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530848 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-certs\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530872 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-token\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530932 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvnf8\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-kube-api-access-pvnf8\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-podinfo\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.530986 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-envoy\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531249 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.531141 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-data\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531667 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.531331 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-credential-socket\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531667 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.531359 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-socket\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531667 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.531451 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-certs\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.531667 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.531551 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3c18502-fb64-4f99-bb93-1bafd11bb517-istiod-ca-cert\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.533488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.533456 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-envoy\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.533488 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.533476 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-podinfo\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.538150 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.538126 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-token\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.538248 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.538182 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvnf8\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-kube-api-access-pvnf8\") pod \"e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.682985 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.682893 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:51.809974 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.809952 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd"] Jul 06 18:18:51.812279 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:18:51.812250 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c18502_fb64_4f99_bb93_1bafd11bb517.slice/crio-beaf0fca93810f59e3926ef41389c37c9cf95d616d9926d8d88456c57e79e7d2 WatchSource:0}: Error finding container beaf0fca93810f59e3926ef41389c37c9cf95d616d9926d8d88456c57e79e7d2: Status 404 returned error can't find the container with id beaf0fca93810f59e3926ef41389c37c9cf95d616d9926d8d88456c57e79e7d2 Jul 06 18:18:51.814397 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.814366 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:51.814518 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.814449 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:51.814518 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:51.814478 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:52.242570 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:52.242527 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" event={"ID":"d3c18502-fb64-4f99-bb93-1bafd11bb517","Type":"ContainerStarted","Data":"5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182"} Jul 06 18:18:52.242750 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:52.242593 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" event={"ID":"d3c18502-fb64-4f99-bb93-1bafd11bb517","Type":"ContainerStarted","Data":"beaf0fca93810f59e3926ef41389c37c9cf95d616d9926d8d88456c57e79e7d2"} Jul 06 18:18:52.258996 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:52.258945 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" podStartSLOduration=1.25893007 podStartE2EDuration="1.25893007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:52.258704652 +0000 UTC m=+2299.664237576" watchObservedRunningTime="2026-07-06 18:18:52.25893007 +0000 UTC m=+2299.664462995" Jul 06 18:18:52.683325 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:52.683292 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:52.688146 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:52.688123 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:53.247726 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:53.247684 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:53.248512 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:53.248491 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:18:55.790862 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.790812 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:55.873381 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873352 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-socket\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873381 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873380 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98cf782b-6ecf-4a7b-9b58-538430707391-istio-podinfo\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873616 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873403 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98cf782b-6ecf-4a7b-9b58-538430707391-istiod-ca-cert\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873616 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873497 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-envoy\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873616 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873525 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n927\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-kube-api-access-9n927\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873616 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873567 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-data\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873616 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873598 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-certs\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873853 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873647 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:55.873853 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873746 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-credential-socket\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.873853 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873799 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-istio-token\") pod \"98cf782b-6ecf-4a7b-9b58-538430707391\" (UID: \"98cf782b-6ecf-4a7b-9b58-538430707391\") " Jul 06 18:18:55.874032 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873858 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98cf782b-6ecf-4a7b-9b58-538430707391-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:18:55.874032 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873863 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:55.874032 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.873985 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:55.874196 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.874174 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.874266 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.874203 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98cf782b-6ecf-4a7b-9b58-538430707391-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.874266 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.874222 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.874266 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.874238 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.874612 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.874592 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-data" (OuterVolumeSpecName: "istio-data") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:55.875759 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.875735 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:18:55.875862 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.875841 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-kube-api-access-9n927" (OuterVolumeSpecName: "kube-api-access-9n927") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "kube-api-access-9n927". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:55.876073 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.876049 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98cf782b-6ecf-4a7b-9b58-538430707391-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:18:55.876159 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.876051 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-istio-token" (OuterVolumeSpecName: "istio-token") pod "98cf782b-6ecf-4a7b-9b58-538430707391" (UID: "98cf782b-6ecf-4a7b-9b58-538430707391"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:18:55.975626 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.975593 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.975626 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.975618 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98cf782b-6ecf-4a7b-9b58-538430707391-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.975626 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.975631 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.975836 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.975642 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n927\" (UniqueName: \"kubernetes.io/projected/98cf782b-6ecf-4a7b-9b58-538430707391-kube-api-access-9n927\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:55.975836 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:55.975651 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98cf782b-6ecf-4a7b-9b58-538430707391-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:18:56.264736 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.264699 2558 generic.go:356] "Generic (PLEG): container finished" podID="98cf782b-6ecf-4a7b-9b58-538430707391" containerID="837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567" exitCode=0 Jul 06 18:18:56.264899 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.264752 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" Jul 06 18:18:56.264899 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.264750 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" event={"ID":"98cf782b-6ecf-4a7b-9b58-538430707391","Type":"ContainerDied","Data":"837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567"} Jul 06 18:18:56.264899 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.264863 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5" event={"ID":"98cf782b-6ecf-4a7b-9b58-538430707391","Type":"ContainerDied","Data":"17ee3b4dfe4e3822693cb134432330da80dc3a07d368228e19149b0812ed064d"} Jul 06 18:18:56.264899 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.264887 2558 scope.go:117] "RemoveContainer" containerID="837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567" Jul 06 18:18:56.273881 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.273861 2558 scope.go:117] "RemoveContainer" containerID="d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa" Jul 06 18:18:56.281352 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.281334 2558 scope.go:117] "RemoveContainer" containerID="837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567" Jul 06 18:18:56.281591 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:56.281572 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567\": container with ID starting with 837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567 not found: ID does not exist" containerID="837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567" Jul 06 18:18:56.281673 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.281602 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567"} err="failed to get container status \"837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567\": rpc error: code = NotFound desc = could not find container \"837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567\": container with ID starting with 837ae9820ece697036e2929f02de93d9137044cb7ff3999e029ee5929795b567 not found: ID does not exist" Jul 06 18:18:56.281673 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.281627 2558 scope.go:117] "RemoveContainer" containerID="d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa" Jul 06 18:18:56.281857 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:18:56.281839 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa\": container with ID starting with d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa not found: ID does not exist" containerID="d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa" Jul 06 18:18:56.281908 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.281863 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa"} err="failed to get container status \"d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa\": rpc error: code = NotFound desc = could not find container \"d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa\": container with ID starting with d01f3156b4d2ccac83695e0ca80c85ed318813ed471f280d2160e2d02dffeaaa not found: ID does not exist" Jul 06 18:18:56.285327 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.285306 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5"] Jul 06 18:18:56.289218 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:56.289196 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-77c307-openshift-default-6866dc8d7f-7b5l5"] Jul 06 18:18:57.070442 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:57.070402 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" path="/var/lib/kubelet/pods/98cf782b-6ecf-4a7b-9b58-538430707391/volumes" Jul 06 18:18:59.194370 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.194328 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v"] Jul 06 18:18:59.194731 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.194678 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" Jul 06 18:18:59.194731 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.194690 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" Jul 06 18:18:59.194731 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.194702 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" Jul 06 18:18:59.194731 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.194709 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed7568ab-4b1e-49f8-96f4-6c192f2617d7" containerName="istio-proxy" Jul 06 18:18:59.194868 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.194777 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" Jul 06 18:18:59.194868 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.194785 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" Jul 06 18:18:59.197585 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.197566 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.199664 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.199644 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-2861c1-openshift-default-dockercfg-xjcxv\"" Jul 06 18:18:59.207384 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.207361 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v"] Jul 06 18:18:59.308047 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308008 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-certs\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308235 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308066 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zknwj\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-kube-api-access-zknwj\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308235 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308099 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istiod-ca-cert\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308235 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308117 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-token\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308235 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308142 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-envoy\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308456 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308276 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-podinfo\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308456 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308317 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-socket\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308456 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308404 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-data\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.308566 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.308487 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-credential-socket\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.409999 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.409961 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-credential-socket\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410178 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410022 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-certs\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410178 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410056 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zknwj\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-kube-api-access-zknwj\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410178 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410080 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istiod-ca-cert\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410178 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410106 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-token\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410178 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-envoy\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410454 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410193 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-podinfo\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410454 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410220 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-socket\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410454 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410256 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-data\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410625 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410515 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-certs\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410625 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410577 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-data\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410728 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410661 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-credential-socket\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410728 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410713 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-socket\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.410913 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.410894 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istiod-ca-cert\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.412471 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.412447 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-envoy\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.412710 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.412687 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-podinfo\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.417010 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.416991 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-token\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.417294 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.417275 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zknwj\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-kube-api-access-zknwj\") pod \"e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.507344 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.507316 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:18:59.661909 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.661882 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v"] Jul 06 18:18:59.663585 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:18:59.663561 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799ffb65_35c6_4fcd_8202_45dc9925ec8f.slice/crio-961cb01132f39b70b1790cbe65d919fe8d4a862708e073961601263b122a2e12 WatchSource:0}: Error finding container 961cb01132f39b70b1790cbe65d919fe8d4a862708e073961601263b122a2e12: Status 404 returned error can't find the container with id 961cb01132f39b70b1790cbe65d919fe8d4a862708e073961601263b122a2e12 Jul 06 18:18:59.666915 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.666480 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:59.666915 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.666558 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:18:59.666915 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:18:59.666599 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:19:00.286830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:00.286796 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" event={"ID":"799ffb65-35c6-4fcd-8202-45dc9925ec8f","Type":"ContainerStarted","Data":"44d520369d1889144ba41f28a3f5c323674ba007c078c150ec5bd4f81c98af1b"} Jul 06 18:19:00.286830 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:00.286847 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" event={"ID":"799ffb65-35c6-4fcd-8202-45dc9925ec8f","Type":"ContainerStarted","Data":"961cb01132f39b70b1790cbe65d919fe8d4a862708e073961601263b122a2e12"} Jul 06 18:19:00.305437 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:00.305375 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podStartSLOduration=1.305360709 podStartE2EDuration="1.305360709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:00.303683504 +0000 UTC m=+2307.709216453" watchObservedRunningTime="2026-07-06 18:19:00.305360709 +0000 UTC m=+2307.710893633" Jul 06 18:19:00.508104 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:00.508070 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:19:00.512917 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:00.512890 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:19:01.292046 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:01.292013 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:19:01.293100 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:01.293085 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:19:09.249516 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:09.249474 2558 patch_prober.go:28] interesting pod/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 18:19:09.249902 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:09.249559 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 18:19:10.338843 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:10.338803 2558 generic.go:356] "Generic (PLEG): container finished" podID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerID="5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182" exitCode=0 Jul 06 18:19:10.339384 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:10.338854 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" event={"ID":"d3c18502-fb64-4f99-bb93-1bafd11bb517","Type":"ContainerDied","Data":"5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182"} Jul 06 18:19:10.339384 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:10.339358 2558 scope.go:117] "RemoveContainer" containerID="5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182" Jul 06 18:19:10.340680 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:10.340640 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:19:10.340785 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:10.340749 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:19:10.340858 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:10.340800 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:19:11.348528 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:11.348481 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" event={"ID":"d3c18502-fb64-4f99-bb93-1bafd11bb517","Type":"ContainerStarted","Data":"23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a"} Jul 06 18:19:11.683979 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:11.683892 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:19:11.684131 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:11.684001 2558 patch_prober.go:28] interesting pod/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 06 18:19:11.684131 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:11.684037 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 06 18:19:12.684218 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:12.684176 2558 patch_prober.go:28] interesting pod/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 06 18:19:12.684649 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:12.684278 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 06 18:19:13.683637 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:13.683596 2558 patch_prober.go:28] interesting pod/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 06 18:19:13.683800 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:13.683676 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 06 18:19:14.683748 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:14.683707 2558 patch_prober.go:28] interesting pod/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 06 18:19:14.684131 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:14.683777 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 06 18:19:15.686656 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:15.686631 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:19:15.687047 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:15.686917 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:19:15.687505 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:15.687489 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:19:16.643151 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:16.642398 2558 patch_prober.go:28] interesting pod/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:41136->10.134.0.39:15021: read: connection reset by peer" start-of-body= Jul 06 18:19:16.643151 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:16.642479 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": read tcp 10.134.0.2:41136->10.134.0.39:15021: read: connection reset by peer" Jul 06 18:19:17.389484 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:17.389452 2558 generic.go:356] "Generic (PLEG): container finished" podID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerID="44d520369d1889144ba41f28a3f5c323674ba007c078c150ec5bd4f81c98af1b" exitCode=0 Jul 06 18:19:17.389920 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:17.389528 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" event={"ID":"799ffb65-35c6-4fcd-8202-45dc9925ec8f","Type":"ContainerDied","Data":"44d520369d1889144ba41f28a3f5c323674ba007c078c150ec5bd4f81c98af1b"} Jul 06 18:19:17.390168 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:17.390151 2558 scope.go:117] "RemoveContainer" containerID="44d520369d1889144ba41f28a3f5c323674ba007c078c150ec5bd4f81c98af1b" Jul 06 18:19:17.391257 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:17.391229 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:19:17.391330 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:17.391313 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:19:17.391379 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:17.391366 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 18:19:18.395806 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:18.395771 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" event={"ID":"799ffb65-35c6-4fcd-8202-45dc9925ec8f","Type":"ContainerStarted","Data":"ec0ee7c25d48053a45dff32fbecddc48eeae9927ae6ca0574f7b720c906f3a2c"} Jul 06 18:19:18.507762 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:18.507723 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:19:18.507952 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:18.507933 2558 patch_prober.go:28] interesting pod/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 18:19:18.508009 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:18.507989 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 18:19:19.508355 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:19.508321 2558 patch_prober.go:28] interesting pod/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 18:19:19.508747 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:19.508374 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 18:19:20.507819 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:20.507778 2558 patch_prober.go:28] interesting pod/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 18:19:20.507992 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:20.507858 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 18:19:21.507656 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:21.507622 2558 patch_prober.go:28] interesting pod/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 18:19:21.508052 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:21.507691 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 18:19:22.507816 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:22.507779 2558 patch_prober.go:28] interesting pod/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 18:19:22.508211 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:22.507838 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 18:19:23.511305 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:23.511266 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:19:23.511794 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:23.511711 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:19:23.512377 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:19:23.512362 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:20:33.125617 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:20:33.125586 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:20:33.129799 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:20:33.129780 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:20:33.130697 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:20:33.130679 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:20:33.134846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:20:33.134832 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:25:01.027480 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:01.027451 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd"] Jul 06 18:25:01.028045 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:01.027752 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" containerID="cri-o://23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a" gracePeriod=30 Jul 06 18:25:06.270034 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.270010 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:25:06.302706 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302656 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-certs\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.302706 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302691 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-podinfo\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.302861 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302741 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvnf8\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-kube-api-access-pvnf8\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.302861 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302775 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-envoy\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.302861 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302823 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3c18502-fb64-4f99-bb93-1bafd11bb517-istiod-ca-cert\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.302861 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302848 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-data\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.303059 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302878 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-credential-socket\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.303059 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302900 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-token\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.303059 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302932 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-socket\") pod \"d3c18502-fb64-4f99-bb93-1bafd11bb517\" (UID: \"d3c18502-fb64-4f99-bb93-1bafd11bb517\") " Jul 06 18:25:06.303059 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.302954 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:06.303270 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.303137 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:06.303270 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.303264 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.303376 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.303281 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.303376 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.303236 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c18502-fb64-4f99-bb93-1bafd11bb517-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:25:06.303505 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.303444 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:06.303917 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.303897 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:06.305107 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.305078 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-kube-api-access-pvnf8" (OuterVolumeSpecName: "kube-api-access-pvnf8") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "kube-api-access-pvnf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:25:06.305236 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.305211 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:06.305417 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.305398 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:25:06.305533 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.305508 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3c18502-fb64-4f99-bb93-1bafd11bb517" (UID: "d3c18502-fb64-4f99-bb93-1bafd11bb517"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:25:06.403723 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.403702 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.403723 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.403721 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.403846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.403731 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvnf8\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-kube-api-access-pvnf8\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.403846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.403740 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.403846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.403748 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3c18502-fb64-4f99-bb93-1bafd11bb517-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.403846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.403755 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:06.403846 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:06.403762 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3c18502-fb64-4f99-bb93-1bafd11bb517-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:07.035964 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.035931 2558 generic.go:356] "Generic (PLEG): container finished" podID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerID="23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a" exitCode=0 Jul 06 18:25:07.036116 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.035975 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" event={"ID":"d3c18502-fb64-4f99-bb93-1bafd11bb517","Type":"ContainerDied","Data":"23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a"} Jul 06 18:25:07.036116 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.035985 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" Jul 06 18:25:07.036116 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.036050 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd" event={"ID":"d3c18502-fb64-4f99-bb93-1bafd11bb517","Type":"ContainerDied","Data":"beaf0fca93810f59e3926ef41389c37c9cf95d616d9926d8d88456c57e79e7d2"} Jul 06 18:25:07.036116 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.036074 2558 scope.go:117] "RemoveContainer" containerID="23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a" Jul 06 18:25:07.045903 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.045876 2558 scope.go:117] "RemoveContainer" containerID="5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182" Jul 06 18:25:07.053935 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.053913 2558 scope.go:117] "RemoveContainer" containerID="23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a" Jul 06 18:25:07.054628 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:25:07.054245 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a\": container with ID starting with 23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a not found: ID does not exist" containerID="23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a" Jul 06 18:25:07.054628 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.054284 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a"} err="failed to get container status \"23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a\": rpc error: code = NotFound desc = could not find container \"23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a\": container with ID starting with 23badd99bb49053d6fa3e9ec716c21b3c01ad2fa21f17181dea3fc269e2e589a not found: ID does not exist" Jul 06 18:25:07.054628 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.054310 2558 scope.go:117] "RemoveContainer" containerID="5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182" Jul 06 18:25:07.055103 ip-10-0-140-115 kubenswrapper[2558]: E0706 18:25:07.054656 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182\": container with ID starting with 5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182 not found: ID does not exist" containerID="5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182" Jul 06 18:25:07.055103 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.054690 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182"} err="failed to get container status \"5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182\": rpc error: code = NotFound desc = could not find container \"5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182\": container with ID starting with 5d547cc9ba2cdc58191f95e5b0cd5cadaaff5e305d4ed961be57c6ef71e61182 not found: ID does not exist" Jul 06 18:25:07.057085 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.057063 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd"] Jul 06 18:25:07.062696 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.062673 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-91cb9f-openshift-default-7c8b85779f-9k7jd"] Jul 06 18:25:07.070700 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.070680 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" path="/var/lib/kubelet/pods/d3c18502-fb64-4f99-bb93-1bafd11bb517/volumes" Jul 06 18:25:07.936271 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.936240 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v"] Jul 06 18:25:07.936673 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:07.936462 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" containerID="cri-o://ec0ee7c25d48053a45dff32fbecddc48eeae9927ae6ca0574f7b720c906f3a2c" gracePeriod=30 Jul 06 18:25:08.513843 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:08.513810 2558 patch_prober.go:28] interesting pod/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 18:25:08.514026 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:08.513873 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 18:25:13.073367 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.073341 2558 generic.go:356] "Generic (PLEG): container finished" podID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerID="ec0ee7c25d48053a45dff32fbecddc48eeae9927ae6ca0574f7b720c906f3a2c" exitCode=0 Jul 06 18:25:13.073750 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.073408 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" event={"ID":"799ffb65-35c6-4fcd-8202-45dc9925ec8f","Type":"ContainerDied","Data":"ec0ee7c25d48053a45dff32fbecddc48eeae9927ae6ca0574f7b720c906f3a2c"} Jul 06 18:25:13.073750 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.073477 2558 scope.go:117] "RemoveContainer" containerID="44d520369d1889144ba41f28a3f5c323674ba007c078c150ec5bd4f81c98af1b" Jul 06 18:25:13.188263 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.188238 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:25:13.262696 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262671 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-envoy\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.262819 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262708 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-data\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.262819 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262734 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-podinfo\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.262819 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262760 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zknwj\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-kube-api-access-zknwj\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.262980 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262822 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istiod-ca-cert\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.262980 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262940 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-certs\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.262980 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262969 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-token\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.263130 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.262993 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-socket\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.263130 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.263024 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-credential-socket\") pod \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\" (UID: \"799ffb65-35c6-4fcd-8202-45dc9925ec8f\") " Jul 06 18:25:13.263228 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.263212 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 18:25:13.263413 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.263391 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:13.263521 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.263399 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istiod-ca-cert\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.263521 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.263399 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:13.263642 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.263588 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:13.263642 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.263626 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:13.264748 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.264730 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 18:25:13.265131 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.265113 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:25:13.265266 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.265247 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 18:25:13.265327 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.265250 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-kube-api-access-zknwj" (OuterVolumeSpecName: "kube-api-access-zknwj") pod "799ffb65-35c6-4fcd-8202-45dc9925ec8f" (UID: "799ffb65-35c6-4fcd-8202-45dc9925ec8f"). InnerVolumeSpecName "kube-api-access-zknwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 18:25:13.364547 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364526 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-envoy\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.364547 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364545 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-data\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.364665 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364553 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-podinfo\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.364665 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364562 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zknwj\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-kube-api-access-zknwj\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.364665 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364571 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-certs\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.364665 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364579 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/799ffb65-35c6-4fcd-8202-45dc9925ec8f-istio-token\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.364665 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364586 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-workload-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.364665 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.364593 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/799ffb65-35c6-4fcd-8202-45dc9925ec8f-credential-socket\") on node \"ip-10-0-140-115.ec2.internal\" DevicePath \"\"" Jul 06 18:25:13.833684 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:13.833661 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9jssv_3dca3e2f-9412-4154-9717-9d1c98560e2b/server/0.log" Jul 06 18:25:14.073884 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:14.073864 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-5hrqw_4f870b77-c67f-4d34-a43a-85d219f60110/copy-manifests/0.log" Jul 06 18:25:14.078609 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:14.078590 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" event={"ID":"799ffb65-35c6-4fcd-8202-45dc9925ec8f","Type":"ContainerDied","Data":"961cb01132f39b70b1790cbe65d919fe8d4a862708e073961601263b122a2e12"} Jul 06 18:25:14.078709 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:14.078627 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v" Jul 06 18:25:14.078709 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:14.078633 2558 scope.go:117] "RemoveContainer" containerID="ec0ee7c25d48053a45dff32fbecddc48eeae9927ae6ca0574f7b720c906f3a2c" Jul 06 18:25:14.081468 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:14.081445 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-5hrqw_4f870b77-c67f-4d34-a43a-85d219f60110/manager/1.log" Jul 06 18:25:14.099865 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:14.099844 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v"] Jul 06 18:25:14.105152 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:14.105134 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-2861c1-openshift-default-67d8fcbc-lbv5v"] Jul 06 18:25:15.071331 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:15.071293 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" path="/var/lib/kubelet/pods/799ffb65-35c6-4fcd-8202-45dc9925ec8f/volumes" Jul 06 18:25:16.874977 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:16.874940 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4b8v7m_6f738c98-8361-4755-8adc-30c53ba0d0df/istio-proxy/0.log" Jul 06 18:25:16.991902 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:16.991871 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nhx92_fbff2003-37be-4e2f-b7bf-4808a220acd0/discovery/0.log" Jul 06 18:25:17.334338 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:17.334308 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cksjz_012664e2-79dd-41c6-80d3-e3a4c9ccb24e/istio-proxy/1.log" Jul 06 18:25:17.682470 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:17.682380 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6575475f-sg844_64a857a2-eb58-4f29-80cf-a2462430c591/router/0.log" Jul 06 18:25:18.022417 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:18.022381 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rvmw8_75ac95be-f892-437d-aa56-10b2659f51bc/main/0.log" Jul 06 18:25:18.238036 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:18.238011 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pwng8_28da37ec-bdae-4ff8-8036-27ce2c9a54e0/main/0.log" Jul 06 18:25:25.790723 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:25.790643 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vtk7l_6f5de882-ee4b-4c48-bd5e-87726ab35b72/global-pull-secret-syncer/0.log" Jul 06 18:25:25.956402 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:25.956378 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s65dx_4879c2c5-4dae-48c7-85b5-f010d8607f1e/konnectivity-agent/0.log" Jul 06 18:25:26.029956 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:26.029936 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-115.ec2.internal_2fe29e2175c9c75acb15d1d4b57872d2/haproxy/0.log" Jul 06 18:25:33.151763 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:33.151739 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:25:33.155929 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:33.155908 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:25:33.162739 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:33.162716 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:25:33.167242 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:33.167220 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z4cwb_64899c8b-c773-4375-a064-80b0de31b920/ovn-acl-logging/0.log" Jul 06 18:25:33.945392 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:33.945370 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-fh97m_ab81ed4a-edf6-4033-a0ca-c4082f990911/cluster-monitoring-operator/0.log" Jul 06 18:25:34.123996 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.123973 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8pg8s_74bf70e9-ba55-454c-bfee-f999658e59ac/node-exporter/0.log" Jul 06 18:25:34.147391 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.147372 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8pg8s_74bf70e9-ba55-454c-bfee-f999658e59ac/kube-rbac-proxy/0.log" Jul 06 18:25:34.172532 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.172515 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8pg8s_74bf70e9-ba55-454c-bfee-f999658e59ac/init-textfile/0.log" Jul 06 18:25:34.459142 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.459069 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_183eab2a-a5fc-4f78-a841-2b2ef302f87b/prometheus/0.log" Jul 06 18:25:34.482373 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.482351 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_183eab2a-a5fc-4f78-a841-2b2ef302f87b/config-reloader/0.log" Jul 06 18:25:34.507295 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.507278 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_183eab2a-a5fc-4f78-a841-2b2ef302f87b/thanos-sidecar/0.log" Jul 06 18:25:34.532885 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.532845 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_183eab2a-a5fc-4f78-a841-2b2ef302f87b/kube-rbac-proxy-web/0.log" Jul 06 18:25:34.557119 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.557098 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_183eab2a-a5fc-4f78-a841-2b2ef302f87b/kube-rbac-proxy/0.log" Jul 06 18:25:34.581325 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.581303 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_183eab2a-a5fc-4f78-a841-2b2ef302f87b/kube-rbac-proxy-thanos/0.log" Jul 06 18:25:34.606158 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.606135 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_183eab2a-a5fc-4f78-a841-2b2ef302f87b/init-config-reloader/0.log" Jul 06 18:25:34.643302 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.643279 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-r2lnw_d73f6985-e01f-4778-96b9-75a27f2cac4d/prometheus-operator/0.log" Jul 06 18:25:34.662835 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:34.662819 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-r2lnw_d73f6985-e01f-4778-96b9-75a27f2cac4d/kube-rbac-proxy/0.log" Jul 06 18:25:36.394239 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394202 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77"] Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394583 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394600 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394617 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394623 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394630 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394641 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394656 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" Jul 06 18:25:36.394687 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394663 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="98cf782b-6ecf-4a7b-9b58-538430707391" containerName="istio-proxy" Jul 06 18:25:36.395261 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394720 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" Jul 06 18:25:36.395261 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394731 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c18502-fb64-4f99-bb93-1bafd11bb517" containerName="istio-proxy" Jul 06 18:25:36.395261 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394737 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" Jul 06 18:25:36.395261 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.394748 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="799ffb65-35c6-4fcd-8202-45dc9925ec8f" containerName="istio-proxy" Jul 06 18:25:36.399394 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.399377 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.401265 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.401247 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q4mx2\"/\"default-dockercfg-qlwpk\"" Jul 06 18:25:36.401824 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.401802 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q4mx2\"/\"kube-root-ca.crt\"" Jul 06 18:25:36.401914 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.401829 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q4mx2\"/\"openshift-service-ca.crt\"" Jul 06 18:25:36.405905 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.405873 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77"] Jul 06 18:25:36.557107 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.557082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-sys\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.557216 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.557118 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-podres\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.557216 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.557141 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-proc\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.557301 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.557243 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-lib-modules\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.557301 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.557293 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnx8w\" (UniqueName: \"kubernetes.io/projected/285cdba9-9e87-49dc-86b2-01c5d6c2706e-kube-api-access-rnx8w\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657653 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657596 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-lib-modules\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657653 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657637 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnx8w\" (UniqueName: \"kubernetes.io/projected/285cdba9-9e87-49dc-86b2-01c5d6c2706e-kube-api-access-rnx8w\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657814 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657659 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-sys\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657814 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657684 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-podres\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657814 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657744 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-sys\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657814 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657782 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-proc\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657999 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657805 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-lib-modules\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657999 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657826 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-podres\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.657999 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.657858 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/285cdba9-9e87-49dc-86b2-01c5d6c2706e-proc\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.664188 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.664159 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnx8w\" (UniqueName: \"kubernetes.io/projected/285cdba9-9e87-49dc-86b2-01c5d6c2706e-kube-api-access-rnx8w\") pod \"perf-node-gather-daemonset-wfx77\" (UID: \"285cdba9-9e87-49dc-86b2-01c5d6c2706e\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.711220 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.711188 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:36.749480 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.749457 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/1.log" Jul 06 18:25:36.760557 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.760531 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-wg42h_a73e9805-1648-4134-9d52-45c8640dcfb7/console-operator/2.log" Jul 06 18:25:36.843316 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:36.843289 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77"] Jul 06 18:25:36.846784 ip-10-0-140-115 kubenswrapper[2558]: W0706 18:25:36.846761 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod285cdba9_9e87_49dc_86b2_01c5d6c2706e.slice/crio-bb006d78449ab8600b18415c438d730a947408f474b35475cbfe1ea42b97bdf6 WatchSource:0}: Error finding container bb006d78449ab8600b18415c438d730a947408f474b35475cbfe1ea42b97bdf6: Status 404 returned error can't find the container with id bb006d78449ab8600b18415c438d730a947408f474b35475cbfe1ea42b97bdf6 Jul 06 18:25:37.182583 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:37.182550 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" event={"ID":"285cdba9-9e87-49dc-86b2-01c5d6c2706e","Type":"ContainerStarted","Data":"c7c8aac33b625d13ac7d609c3209d080a314a59f3433a2f202fb5004b2199759"} Jul 06 18:25:37.182706 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:37.182602 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" event={"ID":"285cdba9-9e87-49dc-86b2-01c5d6c2706e","Type":"ContainerStarted","Data":"bb006d78449ab8600b18415c438d730a947408f474b35475cbfe1ea42b97bdf6"} Jul 06 18:25:37.182706 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:37.182674 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:37.196672 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:37.196598 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" podStartSLOduration=1.196583938 podStartE2EDuration="1.196583938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:37.195207298 +0000 UTC m=+2704.600740241" watchObservedRunningTime="2026-07-06 18:25:37.196583938 +0000 UTC m=+2704.602117179" Jul 06 18:25:37.276607 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:37.276582 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5457b7966d-59cbr_28983549-fc45-4d92-9f2c-c738fcf35d82/console/0.log" Jul 06 18:25:37.317178 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:37.317157 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-jqrq5_b7e9b308-ae61-40eb-83e9-1a54661dc34c/download-server/0.log" Jul 06 18:25:37.912257 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:37.912226 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-b8nzd_e717ae1c-49af-4b4c-83c7-362d14cab49b/volume-data-source-validator/0.log" Jul 06 18:25:39.024809 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:39.024773 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mgxwp_a34e0be6-2c80-4817-80dd-5c4fbc90feb1/dns/0.log" Jul 06 18:25:39.050533 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:39.050506 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mgxwp_a34e0be6-2c80-4817-80dd-5c4fbc90feb1/kube-rbac-proxy/0.log" Jul 06 18:25:39.189240 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:39.189214 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vx2ng_a25767e7-609c-42a7-8b8b-f793ee69d50f/dns-node-resolver/0.log" Jul 06 18:25:39.720136 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:39.720106 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66c479597f-bqxb2_1fd82183-ab7a-4e74-97e0-447e89300893/registry/0.log" Jul 06 18:25:39.737647 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:39.737621 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66c479597f-bqxb2_1fd82183-ab7a-4e74-97e0-447e89300893/registry/1.log" Jul 06 18:25:39.813182 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:39.813163 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-th6lw_014ceecb-1938-4660-b2fa-7c17bb94b782/node-ca/0.log" Jul 06 18:25:40.765557 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:40.765517 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4b8v7m_6f738c98-8361-4755-8adc-30c53ba0d0df/istio-proxy/0.log" Jul 06 18:25:40.947698 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:40.947666 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nhx92_fbff2003-37be-4e2f-b7bf-4808a220acd0/discovery/0.log" Jul 06 18:25:41.031872 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:41.031814 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cksjz_012664e2-79dd-41c6-80d3-e3a4c9ccb24e/istio-proxy/0.log" Jul 06 18:25:41.085484 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:41.085458 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cksjz_012664e2-79dd-41c6-80d3-e3a4c9ccb24e/istio-proxy/1.log" Jul 06 18:25:41.238531 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:41.238512 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6575475f-sg844_64a857a2-eb58-4f29-80cf-a2462430c591/router/0.log" Jul 06 18:25:41.929517 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:41.929483 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jpln9_3a2ef3dd-ede8-498e-ae66-0941952e042a/serve-healthcheck-canary/0.log" Jul 06 18:25:42.478089 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:42.478062 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-cz4mf_7ef13f52-687a-47a1-afdf-a087c1b840f6/insights-operator/0.log" Jul 06 18:25:42.478970 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:42.478951 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-cz4mf_7ef13f52-687a-47a1-afdf-a087c1b840f6/insights-operator/1.log" Jul 06 18:25:42.571600 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:42.571572 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcrz9_9e6ad0b1-1fdd-495d-9fe7-225f1112d537/kube-rbac-proxy/0.log" Jul 06 18:25:42.596239 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:42.596208 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcrz9_9e6ad0b1-1fdd-495d-9fe7-225f1112d537/exporter/0.log" Jul 06 18:25:42.620259 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:42.620236 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcrz9_9e6ad0b1-1fdd-495d-9fe7-225f1112d537/extractor/0.log" Jul 06 18:25:43.197876 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:43.197851 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-wfx77" Jul 06 18:25:45.344018 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:45.343991 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9jssv_3dca3e2f-9412-4154-9717-9d1c98560e2b/server/0.log" Jul 06 18:25:45.487323 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:45.487300 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-5hrqw_4f870b77-c67f-4d34-a43a-85d219f60110/manager/1.log" Jul 06 18:25:45.550140 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:45.550118 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-5hrqw_4f870b77-c67f-4d34-a43a-85d219f60110/manager/0.log" Jul 06 18:25:45.569238 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:45.569218 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-5hrqw_4f870b77-c67f-4d34-a43a-85d219f60110/copy-manifests/0.log" Jul 06 18:25:47.233567 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:47.233539 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7768b55bcf-gzbgc_86e6ca36-58e5-4b2a-b2c4-991fed2ad46c/manager/0.log" Jul 06 18:25:47.656646 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:47.656617 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4b8v7m_6f738c98-8361-4755-8adc-30c53ba0d0df/istio-proxy/0.log" Jul 06 18:25:47.831225 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:47.831192 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nhx92_fbff2003-37be-4e2f-b7bf-4808a220acd0/discovery/0.log" Jul 06 18:25:47.877655 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:47.877626 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cksjz_012664e2-79dd-41c6-80d3-e3a4c9ccb24e/istio-proxy/0.log" Jul 06 18:25:47.936824 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:47.936763 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cksjz_012664e2-79dd-41c6-80d3-e3a4c9ccb24e/istio-proxy/1.log" Jul 06 18:25:48.064931 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:48.064909 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6575475f-sg844_64a857a2-eb58-4f29-80cf-a2462430c591/router/0.log" Jul 06 18:25:49.263403 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:49.263373 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4b8v7m_6f738c98-8361-4755-8adc-30c53ba0d0df/istio-proxy/0.log" Jul 06 18:25:49.432652 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:49.432619 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-nhx92_fbff2003-37be-4e2f-b7bf-4808a220acd0/discovery/0.log" Jul 06 18:25:49.494288 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:49.494267 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cksjz_012664e2-79dd-41c6-80d3-e3a4c9ccb24e/istio-proxy/0.log" Jul 06 18:25:49.550062 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:49.550004 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cksjz_012664e2-79dd-41c6-80d3-e3a4c9ccb24e/istio-proxy/1.log" Jul 06 18:25:49.677186 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:49.677163 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d6575475f-sg844_64a857a2-eb58-4f29-80cf-a2462430c591/router/0.log" Jul 06 18:25:53.845828 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:53.845797 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bqh7_d1dde903-65f8-4917-97c5-6397a9f48c75/kube-multus-additional-cni-plugins/0.log" Jul 06 18:25:53.869516 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:53.869493 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bqh7_d1dde903-65f8-4917-97c5-6397a9f48c75/egress-router-binary-copy/0.log" Jul 06 18:25:53.893860 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:53.893840 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bqh7_d1dde903-65f8-4917-97c5-6397a9f48c75/cni-plugins/0.log" Jul 06 18:25:53.917073 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:53.917048 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bqh7_d1dde903-65f8-4917-97c5-6397a9f48c75/bond-cni-plugin/0.log" Jul 06 18:25:53.940524 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:53.940497 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bqh7_d1dde903-65f8-4917-97c5-6397a9f48c75/routeoverride-cni/0.log" Jul 06 18:25:53.965387 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:53.965364 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bqh7_d1dde903-65f8-4917-97c5-6397a9f48c75/whereabouts-cni-bincopy/0.log" Jul 06 18:25:53.989981 ip-10-0-140-115 kubenswrapper[2558]: I0706 18:25:53.989959 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4bqh7_d1dde903-65f8-4917-97c5-6397a9f48c75/whereabouts-cni/0.log"