Jul 07 12:40:15.923415 ip-10-0-141-76 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 12:40:15.923425 ip-10-0-141-76 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 12:40:15.923432 ip-10-0-141-76 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 12:40:15.923656 ip-10-0-141-76 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 12:40:26.144598 ip-10-0-141-76 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 12:40:26.144614 ip-10-0-141-76 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot be50b6f9cfe942c5b795c12ff3599302 -- Jul 07 12:42:06.536359 ip-10-0-141-76 systemd[1]: Starting Kubernetes Kubelet... Jul 07 12:42:06.921093 ip-10-0-141-76 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:42:06.921093 ip-10-0-141-76 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 12:42:06.921093 ip-10-0-141-76 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:42:06.921093 ip-10-0-141-76 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 12:42:06.921093 ip-10-0-141-76 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 12:42:06.921093 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.918842 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924080 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924097 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924103 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924107 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924111 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924114 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924117 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924120 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924123 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924126 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924129 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924132 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924134 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924137 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924146 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924149 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924152 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924154 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:06.935594 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924157 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924160 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924163 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924167 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924169 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924172 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924175 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924178 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924180 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924183 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924186 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924189 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924192 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924195 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924198 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924200 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924203 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924207 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924210 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924213 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:06.937452 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924215 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924219 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924222 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924225 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924228 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924231 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924234 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924237 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924240 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924242 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924245 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924248 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924252 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924254 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924257 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924260 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924265 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924269 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924273 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924276 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:06.939800 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924279 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924282 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924284 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924287 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924290 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924293 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924295 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924298 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924301 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924304 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924307 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924311 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924314 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924316 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924319 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924321 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924324 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924327 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924331 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924336 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:06.941513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924339 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924342 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924345 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924347 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924350 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924353 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924356 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924359 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924362 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924364 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924368 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924372 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924375 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924378 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924381 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924383 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924386 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924388 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924391 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:06.943636 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924393 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.924396 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927394 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927403 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927407 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927410 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927414 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927417 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927419 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927422 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927425 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927428 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927431 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927434 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927437 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927440 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927442 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927444 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927447 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927453 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:06.945398 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927455 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927458 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927460 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927463 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927465 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927470 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927473 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927478 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927481 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927484 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927487 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927490 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927493 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927497 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927500 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927503 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927506 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927508 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927511 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:06.947853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927513 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927516 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927519 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927522 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927524 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927527 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927530 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927533 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927536 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927539 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927542 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927544 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927549 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927552 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927555 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927557 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927559 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927562 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927564 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927567 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:06.991498 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927569 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927572 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927574 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927577 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927580 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927583 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927586 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927588 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927591 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927594 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927597 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927599 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927602 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927605 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927608 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927610 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927613 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927616 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927618 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927620 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:07.025713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927623 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:06.998473 ip-10-0-141-76 systemd[1]: Started Kubernetes Kubelet. Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927626 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927629 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927632 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927636 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927638 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927641 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927643 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927646 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927648 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927651 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927654 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927656 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927659 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927662 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927664 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927669 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927672 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927674 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:07.282973 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927676 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927680 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.927684 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928112 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928119 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928123 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928126 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928129 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928132 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928136 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928139 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928141 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928144 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928147 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928150 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928153 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928155 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928160 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:07.295958 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928162 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928165 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928167 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928170 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928172 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928175 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928177 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928180 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928183 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928185 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928188 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928191 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928195 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928199 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928202 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928205 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928208 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928211 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928214 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:07.297308 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928216 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928219 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928222 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928224 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928227 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928230 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928232 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928235 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928237 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928240 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928243 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928247 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928250 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928253 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928255 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928258 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928260 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928263 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928265 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928267 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:07.298340 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928270 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928272 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928275 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928277 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928281 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928283 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928286 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928289 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928291 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928294 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928296 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928299 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928301 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928304 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928307 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928310 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928313 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928315 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928318 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928320 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:07.298946 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928323 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928326 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928330 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928333 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928336 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928339 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928341 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928343 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928346 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928349 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928351 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928353 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928356 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928358 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928361 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928363 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928367 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928369 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928372 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928374 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:07.299611 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928377 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928379 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928381 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928384 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.928386 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928464 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928472 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928478 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928483 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928487 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928490 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928495 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928499 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928503 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928506 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928510 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928515 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928518 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928521 2578 flags.go:64] FLAG: --cgroup-root="" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928524 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928527 2578 flags.go:64] FLAG: --client-ca-file="" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928530 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928534 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928538 2578 flags.go:64] FLAG: --cluster-domain="" Jul 07 12:42:07.300267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928540 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928543 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928546 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928550 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928555 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928558 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928561 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928565 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928568 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928570 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928573 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928576 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928580 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928584 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928587 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928590 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928592 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928595 2578 flags.go:64] FLAG: --enable-server="true" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928598 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928603 2578 flags.go:64] FLAG: --event-burst="100" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928606 2578 flags.go:64] FLAG: --event-qps="50" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928609 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928612 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928615 2578 flags.go:64] FLAG: --eviction-hard="" Jul 07 12:42:07.301007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928621 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928624 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928627 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928630 2578 flags.go:64] FLAG: --eviction-soft="" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928633 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928636 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928639 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928642 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928645 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928647 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928650 2578 flags.go:64] FLAG: --feature-gates="" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928653 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928658 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928661 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928664 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928667 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928670 2578 flags.go:64] FLAG: --help="false" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928673 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928676 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928679 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928682 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928685 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928689 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928692 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 12:42:07.301706 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928695 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928697 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928700 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928703 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928706 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928709 2578 flags.go:64] FLAG: --kube-reserved="" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928712 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928714 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928718 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928722 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928725 2578 flags.go:64] FLAG: --lock-file="" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928728 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928731 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928734 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928739 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928742 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928745 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928748 2578 flags.go:64] FLAG: --logging-format="text" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928750 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928754 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928758 2578 flags.go:64] FLAG: --manifest-url="" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928761 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928766 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928769 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928773 2578 flags.go:64] FLAG: --max-pods="110" Jul 07 12:42:07.302578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928776 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928779 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928782 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928785 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928788 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928791 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928794 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928802 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928805 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928807 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928810 2578 flags.go:64] FLAG: --pod-cidr="" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928813 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928819 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928822 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928825 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928828 2578 flags.go:64] FLAG: --port="10250" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928833 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928836 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dd9b3652175649d5" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928839 2578 flags.go:64] FLAG: --qos-reserved="" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928842 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928845 2578 flags.go:64] FLAG: --register-node="true" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928848 2578 flags.go:64] FLAG: --register-with-taints="" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928852 2578 flags.go:64] FLAG: --registry-burst="10" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928855 2578 flags.go:64] FLAG: --registry-qps="5" Jul 07 12:42:07.303356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928857 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928860 2578 flags.go:64] FLAG: --reserved-memory="" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928864 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928868 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928871 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928874 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928876 2578 flags.go:64] FLAG: --runonce="false" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928879 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928882 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928885 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928888 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928891 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928894 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928897 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928900 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928911 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928916 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928919 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928922 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928925 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928928 2578 flags.go:64] FLAG: --system-cgroups="" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928931 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928936 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928939 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928943 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 12:42:07.304164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928947 2578 flags.go:64] FLAG: --tls-min-version="" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928951 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928954 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928957 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928960 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928962 2578 flags.go:64] FLAG: --v="2" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928967 2578 flags.go:64] FLAG: --version="false" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928970 2578 flags.go:64] FLAG: --vmodule="" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928974 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.928977 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929111 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929115 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929118 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929122 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929125 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929127 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929130 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929132 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929135 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929138 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929140 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929143 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:07.304919 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929145 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929148 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929150 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929153 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929156 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929158 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929160 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929163 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929166 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929170 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929172 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929175 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929178 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929181 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929183 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929186 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929189 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929191 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929194 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929197 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929201 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:07.305626 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929203 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929206 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929208 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929211 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929213 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929216 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929218 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929221 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929225 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929228 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929231 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929234 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929237 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929239 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929242 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929247 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929249 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929253 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:07.306257 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929257 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929260 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929264 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929267 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929270 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929272 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929279 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929282 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929285 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929287 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929290 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929292 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929295 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929300 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929303 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929305 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929308 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929311 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929313 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:07.306791 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929316 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929319 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929321 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929324 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929327 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929330 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929333 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929336 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929338 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929341 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929343 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929346 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929348 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929351 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929355 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929360 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929363 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929366 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929368 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929371 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:07.307368 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929374 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929377 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929380 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929382 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929385 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929387 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929392 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929394 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.929397 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.930225 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.938230 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.938251 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.938271 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.938280 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938357 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:07.307980 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938364 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938369 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938373 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938378 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938383 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938387 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938391 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938394 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938400 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938405 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938410 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938414 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938420 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938425 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938430 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938434 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938439 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938443 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938447 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:07.308526 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938451 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938455 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938459 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938462 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938466 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938471 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938475 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938479 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938483 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938487 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938491 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938495 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938499 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938502 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938506 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938510 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938514 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938517 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938521 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938524 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:07.309407 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938528 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938531 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938536 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938539 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938543 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938547 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938551 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938555 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938559 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938562 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938566 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938570 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938574 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938578 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938581 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938585 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938588 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938592 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938596 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938600 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:07.311853 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938604 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938607 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938612 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938615 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938619 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938623 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938627 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938631 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938635 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938638 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938642 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938645 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938649 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938653 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938658 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938661 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938665 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938671 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938675 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938679 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:07.312506 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938683 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938686 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938690 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938695 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938699 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938703 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938706 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938711 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938714 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938718 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938725 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938729 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938735 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938740 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938744 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938749 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938753 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938758 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:07.313158 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938762 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.938769 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938930 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938937 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938941 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938945 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938950 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938954 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938958 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938962 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938966 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938970 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938974 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938977 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 12:42:07.313656 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938983 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938989 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938994 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.938998 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939002 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939006 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939010 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939014 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939018 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939021 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939027 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939031 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939035 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939056 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939061 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939064 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939069 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939073 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939077 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939081 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 12:42:07.314131 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939085 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939089 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939092 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939096 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939099 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939103 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939107 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939111 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939115 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939118 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939122 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939125 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939129 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939133 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939137 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939140 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939144 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939149 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939154 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 12:42:07.314680 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939159 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939164 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939169 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939175 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939179 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939184 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939188 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939192 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939195 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939199 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939204 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939207 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939211 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939215 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939218 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939222 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939226 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939230 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939233 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939237 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 12:42:07.524854 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939241 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939245 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939248 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939252 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939255 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939259 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939263 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939267 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939271 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939275 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939278 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939282 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939286 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939290 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939294 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939299 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939304 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939308 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939311 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939315 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 12:42:07.525468 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939319 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939323 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939326 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939331 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939334 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939338 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939341 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:06.939345 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.939352 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.939522 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.942861 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.944438 2578 server.go:1019] "Starting client certificate rotation" Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.944552 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 12:42:07.526122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.944590 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 12:42:07.526553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.947172 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 12:42:07.526553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.948381 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 12:42:07.526553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.958655 2578 log.go:25] "Validated CRI v1 runtime API" Jul 07 12:42:07.526553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.960961 2578 log.go:25] "Validated CRI v1 image API" Jul 07 12:42:07.526553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.961837 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 12:42:07.526553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.965896 2578 fs.go:135] Filesystem UUIDs: map[1aef54f4-a180-4d04-ac6b-fe359656cfc6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8ec34447-f2d3-4857-859c-704387cc72d3:/dev/nvme0n1p4] Jul 07 12:42:07.526553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.965913 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.971975 2578 manager.go:216] Machine: {Timestamp:2026-07-07 12:42:06.970475933 +0000 UTC m=+0.315130108 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112075 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2136f98a4696f9962b7e177225a754 SystemUUID:ec2136f9-8a46-96f9-962b-7e177225a754 BootID:be50b6f9-cfe9-42c5-b795-c12ff3599302 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3f:75:0d:5f:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3f:75:0d:5f:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:57:1d:c8:9f:e4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.972109 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.972206 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.974925 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.974961 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-76.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.975880 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.975898 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.975908 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.976058 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.977255 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.978960 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982313 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982448 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982477 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982487 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982502 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 12:42:07.527350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982512 2578 kubelet.go:398] "Adding apiserver pod source" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982521 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982898 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.982917 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.985596 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.987999 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.988026 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989398 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989422 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989437 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989449 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989465 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989476 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989491 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989500 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989511 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989643 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989853 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.989866 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.991458 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.991474 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.994788 2578 server.go:1268] "Started kubelet" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.994891 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.995077 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-76.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:06.995192 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-76.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:06.997376 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.997790 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.997873 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.999386 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:06.999835 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.002015 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-76.ec2.internal.18c0026eb7f67048 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-76.ec2.internal,UID:ip-10-0-141-76.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-76.ec2.internal,},FirstTimestamp:2026-07-07 12:42:06.994755656 +0000 UTC m=+0.339409834,LastTimestamp:2026-07-07 12:42:06.994755656 +0000 UTC m=+0.339409834,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-76.ec2.internal,}" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.003703 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.008499 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.008505 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.008578 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009197 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009197 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009231 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009336 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009346 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009480 2578 factory.go:153] Registering CRI-O factory Jul 07 12:42:07.527892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009494 2578 factory.go:223] Registration of the crio container factory successfully Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009542 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009552 2578 factory.go:55] Registering systemd factory Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009560 2578 factory.go:223] Registration of the systemd container factory successfully Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009587 2578 factory.go:103] Registering Raw factory Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009598 2578 manager.go:1196] Started watching for new ooms in manager Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.009996 2578 manager.go:349] Starting recovery of all containers Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.019699 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-76.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.019768 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.020197 2578 manager.go:354] Recovery completed Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.026216 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.028718 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4qv84" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.030011 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.030050 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.030063 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.030546 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.030554 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.030569 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.030632 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.031383 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.031406 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.031417 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.031889 2578 policy_none.go:49] "None policy: Start" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.031903 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.031913 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.032945 2578 policy_none.go:47] "Start" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.036889 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4qv84" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.038000 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-76.ec2.internal.18c0026eba109429 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-76.ec2.internal,UID:ip-10-0-141-76.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-76.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-76.ec2.internal,},FirstTimestamp:2026-07-07 12:42:07.030023209 +0000 UTC m=+0.374677383,LastTimestamp:2026-07-07 12:42:07.030023209 +0000 UTC m=+0.374677383,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-76.ec2.internal,}" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.041134 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-76.ec2.internal.18c0026eba1117ef default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-76.ec2.internal,UID:ip-10-0-141-76.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-76.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-76.ec2.internal,},FirstTimestamp:2026-07-07 12:42:07.030056943 +0000 UTC m=+0.374711118,LastTimestamp:2026-07-07 12:42:07.030056943 +0000 UTC m=+0.374711118,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-76.ec2.internal,}" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.109162 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.121710 2578 manager.go:337] "Starting Device Plugin manager" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.121765 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 12:42:07.529021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.121778 2578 server.go:87] "Starting device plugin registration server" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.122061 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.122074 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.122191 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.122313 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.122322 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.122827 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.122888 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.148971 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.150257 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.150296 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.150320 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.150355 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.153020 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.222677 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.223694 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.223727 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.223741 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.223768 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.227014 2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-76.ec2.internal\" not found" node="ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.234307 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.234330 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-76.ec2.internal\": node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.250794 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal"] Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.250861 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.251764 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.251788 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.251802 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.253418 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.253464 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.254381 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.255236 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.255265 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.255274 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.255298 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.255308 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.255279 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.257349 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.257374 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.258015 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientMemory" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.258051 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.258061 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeHasSufficientPID" Jul 07 12:42:07.529988 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.279366 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.287702 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-76.ec2.internal\" not found" node="ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.290271 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-76.ec2.internal\" not found" node="ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.310597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/920310cc46dd559f052edc7c25ed7454-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal\" (UID: \"920310cc46dd559f052edc7c25ed7454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.310622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/920310cc46dd559f052edc7c25ed7454-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal\" (UID: \"920310cc46dd559f052edc7c25ed7454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.310644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/764707e362efae25fa03b75c43299a59-config\") pod \"kube-apiserver-proxy-ip-10-0-141-76.ec2.internal\" (UID: \"764707e362efae25fa03b75c43299a59\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.379566 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.410910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/920310cc46dd559f052edc7c25ed7454-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal\" (UID: \"920310cc46dd559f052edc7c25ed7454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.410936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/920310cc46dd559f052edc7c25ed7454-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal\" (UID: \"920310cc46dd559f052edc7c25ed7454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.410952 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/764707e362efae25fa03b75c43299a59-config\") pod \"kube-apiserver-proxy-ip-10-0-141-76.ec2.internal\" (UID: \"764707e362efae25fa03b75c43299a59\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.411012 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/920310cc46dd559f052edc7c25ed7454-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal\" (UID: \"920310cc46dd559f052edc7c25ed7454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.411080 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/764707e362efae25fa03b75c43299a59-config\") pod \"kube-apiserver-proxy-ip-10-0-141-76.ec2.internal\" (UID: \"764707e362efae25fa03b75c43299a59\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.411111 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/920310cc46dd559f052edc7c25ed7454-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal\" (UID: \"920310cc46dd559f052edc7c25ed7454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.531127 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.480241 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.692220 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.580872 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.692220 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.591025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.692220 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.595182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" Jul 07 12:42:07.692220 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.681800 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.904376 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.782775 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:07.904376 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.882903 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:08.132522 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.944362 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 12:42:08.132522 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:07.944551 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 12:42:08.132522 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:07.983729 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:08.132522 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.042096 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 12:37:07 +0000 UTC" deadline="2028-02-04 15:16:20.413499287 +0000 UTC" Jul 07 12:42:08.132522 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.042152 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13850h34m12.371351252s" Jul 07 12:42:08.132522 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:08.083965 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-76.ec2.internal\" not found" Jul 07 12:42:08.311216 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.139662 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:08.311216 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.155613 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:08.311216 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.208815 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" Jul 07 12:42:08.311216 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.224785 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 12:42:08.311216 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.225791 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" Jul 07 12:42:08.311216 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.235683 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 12:42:08.530472 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:08.388314 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764707e362efae25fa03b75c43299a59.slice/crio-193c9245079aa6a379f48ef4926e79004f31a1851d33441c0a9d0437145ff542 WatchSource:0}: Error finding container 193c9245079aa6a379f48ef4926e79004f31a1851d33441c0a9d0437145ff542: Status 404 returned error can't find the container with id 193c9245079aa6a379f48ef4926e79004f31a1851d33441c0a9d0437145ff542 Jul 07 12:42:08.530472 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:08.388907 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920310cc46dd559f052edc7c25ed7454.slice/crio-d4f2495c43ec178d5c9f2878d7c33e3f72503fe48190cc7ce7c3620fbf2c7d17 WatchSource:0}: Error finding container d4f2495c43ec178d5c9f2878d7c33e3f72503fe48190cc7ce7c3620fbf2c7d17: Status 404 returned error can't find the container with id d4f2495c43ec178d5c9f2878d7c33e3f72503fe48190cc7ce7c3620fbf2c7d17 Jul 07 12:42:08.530472 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.395216 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:42:08.824651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.609869 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.985638 2578 apiserver.go:52] "Watching apiserver" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.994634 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.995001 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-72ltc","openshift-image-registry/node-ca-s27kt","openshift-multus/multus-849dx","openshift-network-operator/iptables-alerter-bn7g5","openshift-ovn-kubernetes/ovnkube-node-vs62b","kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw","openshift-cluster-node-tuning-operator/tuned-qvpk7","openshift-dns/node-resolver-lwwtx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal","openshift-multus/multus-additional-cni-plugins-56mk4","openshift-multus/network-metrics-daemon-hjcq2","openshift-network-diagnostics/network-check-target-lrs7r","kube-system/global-pull-secret-syncer-22mxz"] Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.997262 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.998621 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:08.998681 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.999701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:08.999923 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nj27s\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.000018 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.000018 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.000278 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-849dx" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.002384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.002733 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.002877 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.002889 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-g5jtt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.002933 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.002942 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.004040 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.005063 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.005083 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlxnj\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.005104 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.005107 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.005396 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.005450 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.006346 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.006544 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.006782 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.006797 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.006996 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.007329 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 12:42:09.026301 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.007372 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.007448 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gfj2n\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.009084 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.009147 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.009601 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9rv6v\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.010886 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.011606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.013017 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.014208 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.014216 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.014217 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wv4kb\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.014567 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.016085 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.016120 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.016166 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.016228 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2hx2h\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.017134 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.017204 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.017353 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-q8jv9\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.017845 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.019812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-ovn\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.019879 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.019922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwslj\" (UniqueName: \"kubernetes.io/projected/71c2d709-99a0-43a4-82c0-632eed5c966a-kube-api-access-mwslj\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.019964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw64f\" (UniqueName: \"kubernetes.io/projected/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-kube-api-access-kw64f\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a56144e2-5d53-4063-8cbd-cb0057bf5483-iptables-alerter-script\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-socket-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-cni-bin\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.028026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020281 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2q22v\"" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020370 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-cni-multus\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020395 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-system-cni-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovnkube-config\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovn-node-metrics-cert\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-multus-certs\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020661 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.020997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmv7l\" (UniqueName: \"kubernetes.io/projected/a56144e2-5d53-4063-8cbd-cb0057bf5483-kube-api-access-cmv7l\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-systemd\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-log-socket\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4df33d13-2cae-46fc-a03f-e104124d7d38-konnectivity-ca\") pod \"konnectivity-agent-72ltc\" (UID: \"4df33d13-2cae-46fc-a03f-e104124d7d38\") " pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-device-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021148 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-socket-dir-parent\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.029100 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-systemd-units\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021248 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4df33d13-2cae-46fc-a03f-e104124d7d38-agent-certs\") pod \"konnectivity-agent-72ltc\" (UID: \"4df33d13-2cae-46fc-a03f-e104124d7d38\") " pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.021263 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-daemon-config\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovnkube-script-lib\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cc952fb-050d-439d-a39a-3167617f0640-dbus\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021400 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021433 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmlmz\" (UniqueName: \"kubernetes.io/projected/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-kube-api-access-mmlmz\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a56144e2-5d53-4063-8cbd-cb0057bf5483-host-slash\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-env-overrides\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cc952fb-050d-439d-a39a-3167617f0640-kubelet-config\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-cni-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-os-release\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-hostroot\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-node-log\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-cni-netd\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-cnibin\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z9bk\" (UniqueName: \"kubernetes.io/projected/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-kube-api-access-9z9bk\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021789 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71c2d709-99a0-43a4-82c0-632eed5c966a-hosts-file\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c2d709-99a0-43a4-82c0-632eed5c966a-tmp-dir\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021851 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-slash\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021901 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-sys-fs\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021945 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-k8s-cni-cncf-io\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-kubelet\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.021993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-etc-kubernetes\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-kubelet\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-run-netns\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-var-lib-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-cni-bin\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-conf-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022178 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-registration-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.030945 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-cni-binary-copy\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.031882 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-netns\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.031882 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.022247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-etc-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.046301 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbvtb" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.055355 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbvtb" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.110877 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-sys\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-cni-multus\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xk7j\" (UniqueName: \"kubernetes.io/projected/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-kube-api-access-7xk7j\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122627 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-cni-multus\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122688 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-cni-binary-copy\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-system-cni-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122775 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovnkube-config\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovn-node-metrics-cert\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123009 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.122971 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-system-cni-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123043 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-modprobe-d\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-multus-certs\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123113 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmv7l\" (UniqueName: \"kubernetes.io/projected/a56144e2-5d53-4063-8cbd-cb0057bf5483-kube-api-access-cmv7l\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.207057 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123273 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-multus-certs\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-systemd\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123371 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-systemd\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123413 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-log-socket\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123390 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-log-socket\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4df33d13-2cae-46fc-a03f-e104124d7d38-konnectivity-ca\") pod \"konnectivity-agent-72ltc\" (UID: \"4df33d13-2cae-46fc-a03f-e104124d7d38\") " pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-device-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysctl-d\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-socket-dir-parent\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.123988 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-device-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124068 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-systemd-units\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4df33d13-2cae-46fc-a03f-e104124d7d38-agent-certs\") pod \"konnectivity-agent-72ltc\" (UID: \"4df33d13-2cae-46fc-a03f-e104124d7d38\") " pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124135 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-socket-dir-parent\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124158 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-systemd-units\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124171 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-host\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.209016 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124201 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovnkube-config\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-run\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124311 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-var-lib-kubelet\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-daemon-config\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124442 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovnkube-script-lib\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cc952fb-050d-439d-a39a-3167617f0640-dbus\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124609 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4df33d13-2cae-46fc-a03f-e104124d7d38-konnectivity-ca\") pod \"konnectivity-agent-72ltc\" (UID: \"4df33d13-2cae-46fc-a03f-e104124d7d38\") " pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmlmz\" (UniqueName: \"kubernetes.io/projected/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-kube-api-access-mmlmz\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-lib-modules\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-os-release\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0cc952fb-050d-439d-a39a-3167617f0640-dbus\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a56144e2-5d53-4063-8cbd-cb0057bf5483-host-slash\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-env-overrides\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cc952fb-050d-439d-a39a-3167617f0640-kubelet-config\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.209722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-cni-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-os-release\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.124947 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-daemon-config\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysconfig\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125044 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-hostroot\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.125063 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125078 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-node-log\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125110 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.125158 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret podName:0cc952fb-050d-439d-a39a-3167617f0640 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:09.625136036 +0000 UTC m=+2.969790221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret") pod "global-pull-secret-syncer-22mxz" (UID: "0cc952fb-050d-439d-a39a-3167617f0640") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125181 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovnkube-script-lib\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-cni-netd\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125196 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-cnibin\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125250 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-cni-netd\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z9bk\" (UniqueName: \"kubernetes.io/projected/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-kube-api-access-9z9bk\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125270 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-env-overrides\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.210716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71c2d709-99a0-43a4-82c0-632eed5c966a-hosts-file\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125282 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125319 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c2d709-99a0-43a4-82c0-632eed5c966a-tmp-dir\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125318 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0cc952fb-050d-439d-a39a-3167617f0640-kubelet-config\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-slash\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125399 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a56144e2-5d53-4063-8cbd-cb0057bf5483-host-slash\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125406 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-hostroot\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125405 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-cnibin\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125416 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-node-log\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-sys-fs\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125431 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-cni-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125478 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71c2d709-99a0-43a4-82c0-632eed5c966a-hosts-file\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-tuned\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125513 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-os-release\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125537 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-sys-fs\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125550 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-slash\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125582 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-k8s-cni-cncf-io\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.211456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125643 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-k8s-cni-cncf-io\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-kubelet\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-kubelet\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-etc-kubernetes\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-kubelet\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71c2d709-99a0-43a4-82c0-632eed5c966a-tmp-dir\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-run-netns\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125789 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-etc-kubernetes\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125832 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-kubelet\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125845 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-run-netns\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-var-lib-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125893 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-cni-bin\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125902 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-var-lib-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-conf-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125942 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-var-lib-cni-bin\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-registration-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-host\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-registration-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.212099 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.125991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kctl8\" (UniqueName: \"kubernetes.io/projected/20c92648-cce2-45f9-891b-e91adcbb8f08-kube-api-access-kctl8\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-multus-conf-dir\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-cnibin\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-cni-binary-copy\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-netns\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126153 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-host-run-netns\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-etc-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126202 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-ovn\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwslj\" (UniqueName: \"kubernetes.io/projected/71c2d709-99a0-43a4-82c0-632eed5c966a-kube-api-access-mwslj\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126250 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-etc-openvswitch\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126260 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-run-ovn\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-kubernetes\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-system-cni-dir\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126323 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw64f\" (UniqueName: \"kubernetes.io/projected/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-kube-api-access-kw64f\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a56144e2-5d53-4063-8cbd-cb0057bf5483-iptables-alerter-script\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.212888 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-systemd\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-984qm\" (UniqueName: \"kubernetes.io/projected/b7392278-af8b-4f18-b906-0632aeda6a07-kube-api-access-984qm\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126575 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f7tm\" (UniqueName: \"kubernetes.io/projected/869f90b8-eca3-4c2f-9d03-db5f9207ae08-kube-api-access-6f7tm\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126689 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-cni-binary-copy\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.126987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a56144e2-5d53-4063-8cbd-cb0057bf5483-iptables-alerter-script\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-socket-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127143 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-serviceca\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127165 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysctl-conf\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127171 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-socket-dir\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20c92648-cce2-45f9-891b-e91adcbb8f08-tmp\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-cni-bin\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127309 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-host-cni-bin\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127647 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-ovn-node-metrics-cert\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.127809 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4df33d13-2cae-46fc-a03f-e104124d7d38-agent-certs\") pod \"konnectivity-agent-72ltc\" (UID: \"4df33d13-2cae-46fc-a03f-e104124d7d38\") " pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.133384 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.133400 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:09.213557 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.133419 2578 projected.go:196] Error preparing data for projected volume kube-api-access-ql5md for pod openshift-network-diagnostics/network-check-target-lrs7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.133464 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md podName:4ab45c9b-b63d-4436-a118-aca64a8af739 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:09.633450027 +0000 UTC m=+2.978104190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ql5md" (UniqueName: "kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md") pod "network-check-target-lrs7r" (UID: "4ab45c9b-b63d-4436-a118-aca64a8af739") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.135328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmv7l\" (UniqueName: \"kubernetes.io/projected/a56144e2-5d53-4063-8cbd-cb0057bf5483-kube-api-access-cmv7l\") pod \"iptables-alerter-bn7g5\" (UID: \"a56144e2-5d53-4063-8cbd-cb0057bf5483\") " pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.135680 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwslj\" (UniqueName: \"kubernetes.io/projected/71c2d709-99a0-43a4-82c0-632eed5c966a-kube-api-access-mwslj\") pod \"node-resolver-lwwtx\" (UID: \"71c2d709-99a0-43a4-82c0-632eed5c966a\") " pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.135741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z9bk\" (UniqueName: \"kubernetes.io/projected/bd577f60-e0c1-48ee-8ff3-f06a0f8684fc-kube-api-access-9z9bk\") pod \"ovnkube-node-vs62b\" (UID: \"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.135932 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw64f\" (UniqueName: \"kubernetes.io/projected/e66e0af9-0fc1-471b-8ea7-3bd868b33d8e-kube-api-access-kw64f\") pod \"multus-849dx\" (UID: \"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e\") " pod="openshift-multus/multus-849dx" Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.136075 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmlmz\" (UniqueName: \"kubernetes.io/projected/261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7-kube-api-access-mmlmz\") pod \"aws-ebs-csi-driver-node-9mpzw\" (UID: \"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.155353 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" event={"ID":"920310cc46dd559f052edc7c25ed7454","Type":"ContainerStarted","Data":"d4f2495c43ec178d5c9f2878d7c33e3f72503fe48190cc7ce7c3620fbf2c7d17"} Jul 07 12:42:09.214380 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.156364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" event={"ID":"764707e362efae25fa03b75c43299a59","Type":"ContainerStarted","Data":"193c9245079aa6a379f48ef4926e79004f31a1851d33441c0a9d0437145ff542"} Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228024 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysctl-d\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-host\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-run\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228114 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-var-lib-kubelet\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228177 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-host\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228207 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-run\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-lib-modules\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228211 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-var-lib-kubelet\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228231 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysctl-d\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228266 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-os-release\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysconfig\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-tuned\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228342 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-os-release\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228351 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-lib-modules\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:09.373885 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228396 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysconfig\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-host\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kctl8\" (UniqueName: \"kubernetes.io/projected/20c92648-cce2-45f9-891b-e91adcbb8f08-kube-api-access-kctl8\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-cnibin\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-kubernetes\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228477 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-host\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-system-cni-dir\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.228497 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-cnibin\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228529 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7392278-af8b-4f18-b906-0632aeda6a07-system-cni-dir\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-systemd\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.228561 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs podName:869f90b8-eca3-4c2f-9d03-db5f9207ae08 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:09.728540918 +0000 UTC m=+3.073195098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs") pod "network-metrics-daemon-hjcq2" (UID: "869f90b8-eca3-4c2f-9d03-db5f9207ae08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-kubernetes\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228597 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-systemd\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-984qm\" (UniqueName: \"kubernetes.io/projected/b7392278-af8b-4f18-b906-0632aeda6a07-kube-api-access-984qm\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.626644 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f7tm\" (UniqueName: \"kubernetes.io/projected/869f90b8-eca3-4c2f-9d03-db5f9207ae08-kube-api-access-6f7tm\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228688 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-serviceca\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysctl-conf\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20c92648-cce2-45f9-891b-e91adcbb8f08-tmp\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-sys\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xk7j\" (UniqueName: \"kubernetes.io/projected/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-kube-api-access-7xk7j\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-cni-binary-copy\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-modprobe-d\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228966 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-modprobe-d\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.228989 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-sys\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.229144 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-sysctl-conf\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.229189 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-serviceca\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.229524 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-cni-binary-copy\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.229548 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.229606 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7392278-af8b-4f18-b906-0632aeda6a07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.230747 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/20c92648-cce2-45f9-891b-e91adcbb8f08-etc-tuned\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.231161 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20c92648-cce2-45f9-891b-e91adcbb8f08-tmp\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.627729 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.237305 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-984qm\" (UniqueName: \"kubernetes.io/projected/b7392278-af8b-4f18-b906-0632aeda6a07-kube-api-access-984qm\") pod \"multus-additional-cni-plugins-56mk4\" (UID: \"b7392278-af8b-4f18-b906-0632aeda6a07\") " pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.237896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f7tm\" (UniqueName: \"kubernetes.io/projected/869f90b8-eca3-4c2f-9d03-db5f9207ae08-kube-api-access-6f7tm\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.241800 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xk7j\" (UniqueName: \"kubernetes.io/projected/3ac46ca0-b6cf-45d8-88ca-cd0553738b14-kube-api-access-7xk7j\") pod \"node-ca-s27kt\" (UID: \"3ac46ca0-b6cf-45d8-88ca-cd0553738b14\") " pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.241899 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kctl8\" (UniqueName: \"kubernetes.io/projected/20c92648-cce2-45f9-891b-e91adcbb8f08-kube-api-access-kctl8\") pod \"tuned-qvpk7\" (UID: \"20c92648-cce2-45f9-891b-e91adcbb8f08\") " pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.310564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.320133 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-849dx" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.320483 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261c0cad_9a9c_4ffc_9f7e_5817e17ff2e7.slice/crio-20e711ede13c187008268dff01baf9ac05de5136a6c270010841126d5ec3f049 WatchSource:0}: Error finding container 20e711ede13c187008268dff01baf9ac05de5136a6c270010841126d5ec3f049: Status 404 returned error can't find the container with id 20e711ede13c187008268dff01baf9ac05de5136a6c270010841126d5ec3f049 Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.328327 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66e0af9_0fc1_471b_8ea7_3bd868b33d8e.slice/crio-07a7f0c1d0496bca9b195c9bc55365725de84835a65998640859acb0373de5cc WatchSource:0}: Error finding container 07a7f0c1d0496bca9b195c9bc55365725de84835a65998640859acb0373de5cc: Status 404 returned error can't find the container with id 07a7f0c1d0496bca9b195c9bc55365725de84835a65998640859acb0373de5cc Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.329025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bn7g5" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.332961 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.339439 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lwwtx" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.344730 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd577f60_e0c1_48ee_8ff3_f06a0f8684fc.slice/crio-aeae57ef8a3bfca464caceeb297e632a5d42457397be39fb52d9046dfe4b04d9 WatchSource:0}: Error finding container aeae57ef8a3bfca464caceeb297e632a5d42457397be39fb52d9046dfe4b04d9: Status 404 returned error can't find the container with id aeae57ef8a3bfca464caceeb297e632a5d42457397be39fb52d9046dfe4b04d9 Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.345874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.352383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s27kt" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.354036 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c2d709_99a0_43a4_82c0_632eed5c966a.slice/crio-4d1954df4370fb09714aa383a891f492279ad696ffbac99e6f06ccf234af7c07 WatchSource:0}: Error finding container 4d1954df4370fb09714aa383a891f492279ad696ffbac99e6f06ccf234af7c07: Status 404 returned error can't find the container with id 4d1954df4370fb09714aa383a891f492279ad696ffbac99e6f06ccf234af7c07 Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.357722 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.358959 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df33d13_2cae_46fc_a03f_e104124d7d38.slice/crio-e7d048459f564b23a1ab8adcbf849a4b3b63bc4824003805d8f158d90a09fe89 WatchSource:0}: Error finding container e7d048459f564b23a1ab8adcbf849a4b3b63bc4824003805d8f158d90a09fe89: Status 404 returned error can't find the container with id e7d048459f564b23a1ab8adcbf849a4b3b63bc4824003805d8f158d90a09fe89 Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.362912 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-56mk4" Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.365463 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac46ca0_b6cf_45d8_88ca_cd0553738b14.slice/crio-67c21dbdb7c113c2f507a943a0eccfa843a68381cda954f0f3986acc0bd53b9e WatchSource:0}: Error finding container 67c21dbdb7c113c2f507a943a0eccfa843a68381cda954f0f3986acc0bd53b9e: Status 404 returned error can't find the container with id 67c21dbdb7c113c2f507a943a0eccfa843a68381cda954f0f3986acc0bd53b9e Jul 07 12:42:09.628533 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.368756 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20c92648_cce2_45f9_891b_e91adcbb8f08.slice/crio-f4689a10d6a9a2d7eb7f3d2dbfecc44bc966dd86bdb921066c0c5f3814de0896 WatchSource:0}: Error finding container f4689a10d6a9a2d7eb7f3d2dbfecc44bc966dd86bdb921066c0c5f3814de0896: Status 404 returned error can't find the container with id f4689a10d6a9a2d7eb7f3d2dbfecc44bc966dd86bdb921066c0c5f3814de0896 Jul 07 12:42:09.629532 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:09.378791 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7392278_af8b_4f18_b906_0632aeda6a07.slice/crio-03bfbb9df0637fbaeb1db58438e925b6c2bb7173b9219bb7bd5993addad5df86 WatchSource:0}: Error finding container 03bfbb9df0637fbaeb1db58438e925b6c2bb7173b9219bb7bd5993addad5df86: Status 404 returned error can't find the container with id 03bfbb9df0637fbaeb1db58438e925b6c2bb7173b9219bb7bd5993addad5df86 Jul 07 12:42:09.629532 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.502268 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 12:42:09.632664 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.632623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:09.632893 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.632821 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:09.633019 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.632901 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret podName:0cc952fb-050d-439d-a39a-3167617f0640 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:10.632879628 +0000 UTC m=+3.977533795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret") pod "global-pull-secret-syncer-22mxz" (UID: "0cc952fb-050d-439d-a39a-3167617f0640") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:09.733608 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.733568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:09.733807 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:09.733660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:09.733807 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.733780 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:09.733923 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.733842 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs podName:869f90b8-eca3-4c2f-9d03-db5f9207ae08 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:10.733822457 +0000 UTC m=+4.078476624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs") pod "network-metrics-daemon-hjcq2" (UID: "869f90b8-eca3-4c2f-9d03-db5f9207ae08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:09.733923 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.733779 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:09.734037 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.733945 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:09.734037 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.733964 2578 projected.go:196] Error preparing data for projected volume kube-api-access-ql5md for pod openshift-network-diagnostics/network-check-target-lrs7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:09.734037 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:09.734036 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md podName:4ab45c9b-b63d-4436-a118-aca64a8af739 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:10.734018098 +0000 UTC m=+4.078672278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ql5md" (UniqueName: "kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md") pod "network-check-target-lrs7r" (UID: "4ab45c9b-b63d-4436-a118-aca64a8af739") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:10.057779 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.057665 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:37:09 +0000 UTC" deadline="2028-03-18 18:58:05.947553344 +0000 UTC" Jul 07 12:42:10.057779 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.057716 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14886h15m55.889843381s" Jul 07 12:42:10.151221 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.151186 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:10.151471 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.151332 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:10.151788 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.151762 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:10.151887 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.151867 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:10.151948 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.151933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:10.152060 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.152026 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:10.175197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.175084 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerStarted","Data":"03bfbb9df0637fbaeb1db58438e925b6c2bb7173b9219bb7bd5993addad5df86"} Jul 07 12:42:10.177988 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.177937 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" event={"ID":"20c92648-cce2-45f9-891b-e91adcbb8f08","Type":"ContainerStarted","Data":"f4689a10d6a9a2d7eb7f3d2dbfecc44bc966dd86bdb921066c0c5f3814de0896"} Jul 07 12:42:10.181349 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.181319 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bn7g5" event={"ID":"a56144e2-5d53-4063-8cbd-cb0057bf5483","Type":"ContainerStarted","Data":"5ea8659cbc0e3ac9cb8178f60ea95f883d0e083c1f839322f099d28b1404f736"} Jul 07 12:42:10.187761 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.187734 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" event={"ID":"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7","Type":"ContainerStarted","Data":"20e711ede13c187008268dff01baf9ac05de5136a6c270010841126d5ec3f049"} Jul 07 12:42:10.193529 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.193501 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s27kt" event={"ID":"3ac46ca0-b6cf-45d8-88ca-cd0553738b14","Type":"ContainerStarted","Data":"67c21dbdb7c113c2f507a943a0eccfa843a68381cda954f0f3986acc0bd53b9e"} Jul 07 12:42:10.198008 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.197979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-72ltc" event={"ID":"4df33d13-2cae-46fc-a03f-e104124d7d38","Type":"ContainerStarted","Data":"e7d048459f564b23a1ab8adcbf849a4b3b63bc4824003805d8f158d90a09fe89"} Jul 07 12:42:10.204148 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.204109 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwwtx" event={"ID":"71c2d709-99a0-43a4-82c0-632eed5c966a","Type":"ContainerStarted","Data":"4d1954df4370fb09714aa383a891f492279ad696ffbac99e6f06ccf234af7c07"} Jul 07 12:42:10.212519 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.212488 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"aeae57ef8a3bfca464caceeb297e632a5d42457397be39fb52d9046dfe4b04d9"} Jul 07 12:42:10.233129 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.232857 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-849dx" event={"ID":"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e","Type":"ContainerStarted","Data":"07a7f0c1d0496bca9b195c9bc55365725de84835a65998640859acb0373de5cc"} Jul 07 12:42:10.643635 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.643593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:10.643846 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.643805 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:10.643918 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.643870 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret podName:0cc952fb-050d-439d-a39a-3167617f0640 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:12.643851294 +0000 UTC m=+5.988505478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret") pod "global-pull-secret-syncer-22mxz" (UID: "0cc952fb-050d-439d-a39a-3167617f0640") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:10.744836 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.744795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:10.745215 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:10.744855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:10.745215 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.745000 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:10.745215 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.745021 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:10.745215 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.745034 2578 projected.go:196] Error preparing data for projected volume kube-api-access-ql5md for pod openshift-network-diagnostics/network-check-target-lrs7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:10.745215 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.745117 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md podName:4ab45c9b-b63d-4436-a118-aca64a8af739 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:12.745088799 +0000 UTC m=+6.089742961 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ql5md" (UniqueName: "kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md") pod "network-check-target-lrs7r" (UID: "4ab45c9b-b63d-4436-a118-aca64a8af739") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:10.745561 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.745542 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:10.745657 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:10.745603 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs podName:869f90b8-eca3-4c2f-9d03-db5f9207ae08 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:12.745585902 +0000 UTC m=+6.090240067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs") pod "network-metrics-daemon-hjcq2" (UID: "869f90b8-eca3-4c2f-9d03-db5f9207ae08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:11.058263 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:11.058146 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 12:37:09 +0000 UTC" deadline="2028-01-03 07:29:57.799398856 +0000 UTC" Jul 07 12:42:11.058263 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:11.058209 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13074h47m46.741193733s" Jul 07 12:42:12.151470 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:12.151420 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:12.151979 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.151546 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:12.151979 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:12.151959 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:12.152132 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:12.151962 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:12.152297 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.152210 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:12.152297 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.152076 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:12.665136 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:12.664500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:12.665136 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.664699 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:12.665136 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.664762 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret podName:0cc952fb-050d-439d-a39a-3167617f0640 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:16.664743312 +0000 UTC m=+10.009397479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret") pod "global-pull-secret-syncer-22mxz" (UID: "0cc952fb-050d-439d-a39a-3167617f0640") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:12.765587 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:12.765542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:12.765757 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:12.765641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:12.765818 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.765780 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:12.765886 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.765842 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs podName:869f90b8-eca3-4c2f-9d03-db5f9207ae08 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:16.765823021 +0000 UTC m=+10.110477186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs") pod "network-metrics-daemon-hjcq2" (UID: "869f90b8-eca3-4c2f-9d03-db5f9207ae08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:12.766298 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.766275 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:12.766298 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.766301 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:12.766465 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.766315 2578 projected.go:196] Error preparing data for projected volume kube-api-access-ql5md for pod openshift-network-diagnostics/network-check-target-lrs7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:12.766465 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:12.766365 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md podName:4ab45c9b-b63d-4436-a118-aca64a8af739 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:16.766349016 +0000 UTC m=+10.111003182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ql5md" (UniqueName: "kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md") pod "network-check-target-lrs7r" (UID: "4ab45c9b-b63d-4436-a118-aca64a8af739") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:14.152034 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:14.151289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:14.152034 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:14.151438 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:14.152034 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:14.151510 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:14.152034 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:14.151576 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:14.152034 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:14.151620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:14.152034 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:14.151688 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:16.150874 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:16.150775 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:16.150874 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:16.150824 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:16.150874 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:16.150824 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:16.151442 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.150930 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:16.151442 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.151061 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:16.151538 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.151508 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:16.698341 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:16.698272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:16.698524 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.698436 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:16.698524 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.698520 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret podName:0cc952fb-050d-439d-a39a-3167617f0640 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:24.698497555 +0000 UTC m=+18.043151740 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret") pod "global-pull-secret-syncer-22mxz" (UID: "0cc952fb-050d-439d-a39a-3167617f0640") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:16.799103 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:16.799068 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:16.799294 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:16.799124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:16.799294 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.799256 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:16.799382 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.799328 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs podName:869f90b8-eca3-4c2f-9d03-db5f9207ae08 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:24.799312164 +0000 UTC m=+18.143966347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs") pod "network-metrics-daemon-hjcq2" (UID: "869f90b8-eca3-4c2f-9d03-db5f9207ae08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:16.799382 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.799265 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:16.799382 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.799371 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:16.799382 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.799383 2578 projected.go:196] Error preparing data for projected volume kube-api-access-ql5md for pod openshift-network-diagnostics/network-check-target-lrs7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:16.799577 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:16.799431 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md podName:4ab45c9b-b63d-4436-a118-aca64a8af739 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:24.799414743 +0000 UTC m=+18.144068906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ql5md" (UniqueName: "kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md") pod "network-check-target-lrs7r" (UID: "4ab45c9b-b63d-4436-a118-aca64a8af739") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:18.151414 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:18.150844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:18.151414 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:18.150883 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:18.151414 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:18.150973 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:18.151414 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:18.151267 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:18.151414 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:18.150846 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:18.151414 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:18.151362 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:20.151526 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:20.151491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:20.152013 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:20.151486 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:20.152013 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:20.151632 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:20.152013 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:20.151486 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:20.152013 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:20.151660 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:20.152013 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:20.151767 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:22.150838 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:22.150802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:22.151306 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:22.150802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:22.151306 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:22.150974 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:22.151306 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:22.150802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:22.151306 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:22.151064 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:22.151306 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:22.151106 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:24.151269 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:24.151228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:24.151772 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:24.151228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:24.151772 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.151352 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:24.151772 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:24.151228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:24.151772 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.151434 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:24.151772 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.151519 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:24.763808 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:24.763765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:24.763986 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.763915 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:24.764031 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.763991 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret podName:0cc952fb-050d-439d-a39a-3167617f0640 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:40.763973606 +0000 UTC m=+34.108627775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret") pod "global-pull-secret-syncer-22mxz" (UID: "0cc952fb-050d-439d-a39a-3167617f0640") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:24.864340 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:24.864296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:24.864526 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:24.864367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:24.864526 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.864421 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:24.864526 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.864471 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:24.864526 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.864488 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:24.864526 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.864494 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs podName:869f90b8-eca3-4c2f-9d03-db5f9207ae08 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:40.864474744 +0000 UTC m=+34.209128922 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs") pod "network-metrics-daemon-hjcq2" (UID: "869f90b8-eca3-4c2f-9d03-db5f9207ae08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:24.864526 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.864501 2578 projected.go:196] Error preparing data for projected volume kube-api-access-ql5md for pod openshift-network-diagnostics/network-check-target-lrs7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:24.864785 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:24.864535 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md podName:4ab45c9b-b63d-4436-a118-aca64a8af739 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:40.864523779 +0000 UTC m=+34.209177948 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ql5md" (UniqueName: "kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md") pod "network-check-target-lrs7r" (UID: "4ab45c9b-b63d-4436-a118-aca64a8af739") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:26.151105 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:26.151064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:26.151105 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:26.151103 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:26.151594 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:26.151064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:26.151594 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:26.151194 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:26.151594 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:26.151283 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:26.151594 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:26.151350 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:28.151373 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:28.151329 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:28.151373 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:28.151360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:28.151891 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:28.151345 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:28.151891 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:28.151460 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:28.151891 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:28.151580 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:28.151891 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:28.151662 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:30.151356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.151118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:30.152122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.151139 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:30.152122 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:30.151453 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:30.152122 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:30.151554 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:30.152122 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.151149 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:30.152122 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:30.151639 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:30.280002 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.279968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" event={"ID":"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7","Type":"ContainerStarted","Data":"635ea4e46f38912ef252e5ba460357f98c05f862a95b42e31601919e64daba04"} Jul 07 12:42:30.287590 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.287101 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s27kt" event={"ID":"3ac46ca0-b6cf-45d8-88ca-cd0553738b14","Type":"ContainerStarted","Data":"f6c9ddface44759dfc2af916bde8b3b3143fa7b55b822dd54cd927dbab2bf3f2"} Jul 07 12:42:30.289599 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.289031 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-72ltc" event={"ID":"4df33d13-2cae-46fc-a03f-e104124d7d38","Type":"ContainerStarted","Data":"63feecd2cc726691afd9896d0ea1350563d46d99e6b51f04947816f2afc13b43"} Jul 07 12:42:30.290641 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.290620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lwwtx" event={"ID":"71c2d709-99a0-43a4-82c0-632eed5c966a","Type":"ContainerStarted","Data":"5c01fce941156243ac0caf721cc704dc81012de6e633711524ff0078dbbcf033"} Jul 07 12:42:30.293462 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.293441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"32cbd420cd9937589eab886784a60203c41aaf8d6d76411b979ac58613bf7be3"} Jul 07 12:42:30.293548 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.293495 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"cd4678df19477722504a6b2e2cd54fc09de4511d1ebd2f319c2a189eea6b3760"} Jul 07 12:42:30.293548 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.293526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"f0ac2fbb8556095508044abf857ecefb05ba89dff3978a5c61b51cbceddab37f"} Jul 07 12:42:30.293618 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.293554 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"866fce06cdf03827fb5c19c87ac1def7f2ee694105895b8812aeeda7c455857a"} Jul 07 12:42:30.293618 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.293582 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"6b126cdf2be156dd3adf946ce8bd107f6d9a9630bd28eba3f51b3eb589766ce7"} Jul 07 12:42:30.293688 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.293621 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"4744487dd855e09ef906b5ea341c4ccfbd0ef92c78f7d1feff3d588a048cacd7"} Jul 07 12:42:30.295129 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.295106 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-849dx" event={"ID":"e66e0af9-0fc1-471b-8ea7-3bd868b33d8e","Type":"ContainerStarted","Data":"66105b50388eaefb52fe0d4df98da723a22b0fc1a61d24e64627760abdd0f681"} Jul 07 12:42:30.296553 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.296530 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" event={"ID":"764707e362efae25fa03b75c43299a59","Type":"ContainerStarted","Data":"6e0280c603bde2211e94e46a92b1c73740fe436cf505581f5d1f0042e92112ee"} Jul 07 12:42:30.297816 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.297796 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerStarted","Data":"b5f4901ea2bf0528751b65e24ce6a53a87a67a55513a657c4e5d9ab02c04c678"} Jul 07 12:42:30.299013 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.298994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" event={"ID":"20c92648-cce2-45f9-891b-e91adcbb8f08","Type":"ContainerStarted","Data":"251d9a884c6b01ccb9de34ce12e9535d77f140d0171d258e44deb22c94e76ee8"} Jul 07 12:42:30.322878 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.322829 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s27kt" podStartSLOduration=7.097438344 podStartE2EDuration="23.322811282s" totalImagesPullingTime="16.225372938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:30.306831675 +0000 UTC m=+23.651485860" watchObservedRunningTime="2026-07-07 12:42:30.322811282 +0000 UTC m=+23.667465520" Jul 07 12:42:30.323029 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.322910 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-849dx" podStartSLOduration=3.12644449 podStartE2EDuration="23.322905333s" totalImagesPullingTime="20.196460843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:30.322780146 +0000 UTC m=+23.667434327" watchObservedRunningTime="2026-07-07 12:42:30.322905333 +0000 UTC m=+23.667559517" Jul 07 12:42:30.378251 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.378186 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-72ltc" podStartSLOduration=3.276196743 podStartE2EDuration="23.378165422s" totalImagesPullingTime="20.101968679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:30.338532311 +0000 UTC m=+23.683186495" watchObservedRunningTime="2026-07-07 12:42:30.378165422 +0000 UTC m=+23.722819607" Jul 07 12:42:30.401994 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.401818 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lwwtx" podStartSLOduration=3.294854733 podStartE2EDuration="23.401802265s" totalImagesPullingTime="20.106947532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:30.401412385 +0000 UTC m=+23.746066570" watchObservedRunningTime="2026-07-07 12:42:30.401802265 +0000 UTC m=+23.746456448" Jul 07 12:42:30.430383 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:30.430326 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qvpk7" podStartSLOduration=3.3409454370000002 podStartE2EDuration="23.430308524s" totalImagesPullingTime="20.089363087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:30.430307892 +0000 UTC m=+23.774962077" watchObservedRunningTime="2026-07-07 12:42:30.430308524 +0000 UTC m=+23.774962708" Jul 07 12:42:31.303382 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.303286 2578 generic.go:356] "Generic (PLEG): container finished" podID="920310cc46dd559f052edc7c25ed7454" containerID="41a1a1d76f6e0e6b7767a90ae4c514dbcfb22e3d9a43aa448b7923177bb313b6" exitCode=0 Jul 07 12:42:31.303891 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.303384 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" event={"ID":"920310cc46dd559f052edc7c25ed7454","Type":"ContainerDied","Data":"41a1a1d76f6e0e6b7767a90ae4c514dbcfb22e3d9a43aa448b7923177bb313b6"} Jul 07 12:42:31.305268 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.305238 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7392278-af8b-4f18-b906-0632aeda6a07" containerID="b5f4901ea2bf0528751b65e24ce6a53a87a67a55513a657c4e5d9ab02c04c678" exitCode=0 Jul 07 12:42:31.305393 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.305319 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerDied","Data":"b5f4901ea2bf0528751b65e24ce6a53a87a67a55513a657c4e5d9ab02c04c678"} Jul 07 12:42:31.307409 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.307383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bn7g5" event={"ID":"a56144e2-5d53-4063-8cbd-cb0057bf5483","Type":"ContainerStarted","Data":"4ae5b71b57e854e5eb1928713019ef146c5c1ce4d0adc27ebe4442ea3e716da9"} Jul 07 12:42:31.319726 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.319676 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-76.ec2.internal" podStartSLOduration=23.319658018 podStartE2EDuration="23.319658018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:30.45014943 +0000 UTC m=+23.794803614" watchObservedRunningTime="2026-07-07 12:42:31.319658018 +0000 UTC m=+24.664312203" Jul 07 12:42:31.357795 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.357736 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bn7g5" podStartSLOduration=8.10832713 podStartE2EDuration="24.357714817s" totalImagesPullingTime="16.249387687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:31.356784449 +0000 UTC m=+24.701438635" watchObservedRunningTime="2026-07-07 12:42:31.357714817 +0000 UTC m=+24.702369002" Jul 07 12:42:31.761833 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.761802 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 12:42:31.928922 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.928842 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:31.929401 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:31.929384 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:32.143713 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.143611 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T12:42:31.761825444Z","UUID":"e96b7aac-933e-4e71-8e05-cf7892a70770","Handler":null,"Name":"","Endpoint":""} Jul 07 12:42:32.145032 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.145016 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 12:42:32.145143 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.145071 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 12:42:32.150532 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.150507 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:32.150652 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.150535 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:32.150652 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.150603 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:32.150652 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:32.150619 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:32.150803 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:32.150679 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:32.150887 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:32.150846 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:32.313054 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.312999 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" event={"ID":"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7","Type":"ContainerStarted","Data":"a8b62a7b270a6febca1739a9c11b1b95f17e7c69b73e9bd89d30ab06ed33bb83"} Jul 07 12:42:32.315113 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.315014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" event={"ID":"920310cc46dd559f052edc7c25ed7454","Type":"ContainerStarted","Data":"d0c8c6bdf17a128517d8ddd484e4817d49735ec9c5119f9e57cebde8f2fd556f"} Jul 07 12:42:32.318980 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.318944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"0c6397cc710bcc02bf0a6cac764e408f6a7dccb9a8604ac268566f22a91f7769"} Jul 07 12:42:32.319247 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.319227 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:32.319782 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.319765 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-72ltc" Jul 07 12:42:32.331571 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:32.331519 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-76.ec2.internal" podStartSLOduration=24.331502038 podStartE2EDuration="24.331502038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:32.331478724 +0000 UTC m=+25.676132910" watchObservedRunningTime="2026-07-07 12:42:32.331502038 +0000 UTC m=+25.676156203" Jul 07 12:42:33.323882 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:33.323791 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" event={"ID":"261c0cad-9a9c-4ffc-9f7e-5817e17ff2e7","Type":"ContainerStarted","Data":"6dfe1dd1f5a155018f84cafb70fc400f699a5263c05678ef0cd4d7ca66d11752"} Jul 07 12:42:33.348673 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:33.348620 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mpzw" podStartSLOduration=3.617322098 podStartE2EDuration="26.34859822s" totalImagesPullingTime="22.731276122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:33.348191161 +0000 UTC m=+26.692845347" watchObservedRunningTime="2026-07-07 12:42:33.34859822 +0000 UTC m=+26.693252406" Jul 07 12:42:34.150575 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:34.150536 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:34.150741 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:34.150542 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:34.150741 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:34.150645 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:34.150741 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:34.150537 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:34.150741 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:34.150726 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:34.150873 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:34.150847 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:35.332690 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:35.332486 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" event={"ID":"bd577f60-e0c1-48ee-8ff3-f06a0f8684fc","Type":"ContainerStarted","Data":"9b135cc993ecfe5b3c01a2f314c78a5b65fe81ee34f807b4703e53fce9624464"} Jul 07 12:42:35.333319 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:35.332739 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:35.333319 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:35.332895 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:35.348499 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:35.348468 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:35.358870 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:35.358821 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" podStartSLOduration=8.179875811 podStartE2EDuration="28.358807702s" totalImagesPullingTime="20.178931891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:35.35737935 +0000 UTC m=+28.702033535" watchObservedRunningTime="2026-07-07 12:42:35.358807702 +0000 UTC m=+28.703461886" Jul 07 12:42:36.151056 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.151001 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:36.151217 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.151150 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:36.151217 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:36.151158 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:36.151217 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.151190 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:36.151363 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:36.151296 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:36.151445 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:36.151420 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:36.335877 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.335842 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:36.352892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.352862 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:42:36.442026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.441947 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjcq2"] Jul 07 12:42:36.442197 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.442090 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:36.442266 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:36.442202 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:36.445499 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.445471 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-22mxz"] Jul 07 12:42:36.445657 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.445577 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:36.445720 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:36.445677 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:36.446066 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.446019 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lrs7r"] Jul 07 12:42:36.446405 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:36.446113 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:36.446405 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:36.446196 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:38.151447 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:38.151260 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:38.151872 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:38.151306 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:38.151872 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:38.151520 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:38.151872 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:38.151306 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:38.151872 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:38.151591 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:38.151872 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:38.151684 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:38.342366 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:38.342332 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7392278-af8b-4f18-b906-0632aeda6a07" containerID="05bec401ae64a03b35375c9b71b9dc9e892711f136da494df691f99e14cc38d9" exitCode=0 Jul 07 12:42:38.342538 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:38.342404 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerDied","Data":"05bec401ae64a03b35375c9b71b9dc9e892711f136da494df691f99e14cc38d9"} Jul 07 12:42:40.150549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.150507 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:40.151202 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.150507 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:40.151202 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.150623 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:40.151202 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.150507 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:40.151202 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.150741 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:40.151202 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.150810 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:40.349310 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.349273 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7392278-af8b-4f18-b906-0632aeda6a07" containerID="7031a77bc0ad9e2beb63cdd6d68318b0e93424e192a4c359c142c470702e52e3" exitCode=0 Jul 07 12:42:40.349490 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.349360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerDied","Data":"7031a77bc0ad9e2beb63cdd6d68318b0e93424e192a4c359c142c470702e52e3"} Jul 07 12:42:40.778846 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.778804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:40.779006 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.778918 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:40.779006 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.778967 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret podName:0cc952fb-050d-439d-a39a-3167617f0640 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:12.778954297 +0000 UTC m=+66.123608459 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret") pod "global-pull-secret-syncer-22mxz" (UID: "0cc952fb-050d-439d-a39a-3167617f0640") : object "kube-system"/"original-pull-secret" not registered Jul 07 12:42:40.879413 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.879386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:40.879614 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:40.879425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:40.879614 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.879529 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 12:42:40.879614 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.879531 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:40.879614 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.879542 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 12:42:40.879614 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.879552 2578 projected.go:196] Error preparing data for projected volume kube-api-access-ql5md for pod openshift-network-diagnostics/network-check-target-lrs7r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:40.879614 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.879599 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs podName:869f90b8-eca3-4c2f-9d03-db5f9207ae08 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:12.879581431 +0000 UTC m=+66.224235596 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs") pod "network-metrics-daemon-hjcq2" (UID: "869f90b8-eca3-4c2f-9d03-db5f9207ae08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 12:42:40.879894 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:40.879620 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md podName:4ab45c9b-b63d-4436-a118-aca64a8af739 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:12.879610273 +0000 UTC m=+66.224264434 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ql5md" (UniqueName: "kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md") pod "network-check-target-lrs7r" (UID: "4ab45c9b-b63d-4436-a118-aca64a8af739") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 12:42:41.354358 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:41.354266 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7392278-af8b-4f18-b906-0632aeda6a07" containerID="22a850c191497ce460c412eb380be58334d5a285dd4b72b0abbc3eb873d734ea" exitCode=0 Jul 07 12:42:41.354358 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:41.354328 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerDied","Data":"22a850c191497ce460c412eb380be58334d5a285dd4b72b0abbc3eb873d734ea"} Jul 07 12:42:42.151182 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:42.151144 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:42.151342 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:42.151213 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:42.151342 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:42.151223 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:42.151342 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:42.151291 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lrs7r" podUID="4ab45c9b-b63d-4436-a118-aca64a8af739" Jul 07 12:42:42.151479 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:42.151361 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjcq2" podUID="869f90b8-eca3-4c2f-9d03-db5f9207ae08" Jul 07 12:42:42.151479 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:42.151417 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-22mxz" podUID="0cc952fb-050d-439d-a39a-3167617f0640" Jul 07 12:42:43.414006 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.413840 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-76.ec2.internal" event="NodeReady" Jul 07 12:42:43.414448 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.414205 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 12:42:43.450169 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.450133 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8"] Jul 07 12:42:43.452448 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.452418 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.455182 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.455006 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cd865c99-qfscz"] Jul 07 12:42:43.456103 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.456076 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 12:42:43.456103 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.456100 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sz4f2\"" Jul 07 12:42:43.456282 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.456245 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.456533 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.456514 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.456645 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.456627 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 12:42:43.457680 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.457663 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8"] Jul 07 12:42:43.457801 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.457786 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.459589 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.459571 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll"] Jul 07 12:42:43.459725 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.459702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" Jul 07 12:42:43.461059 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.461024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 12:42:43.461161 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.461089 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8ss65\"" Jul 07 12:42:43.461375 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.461275 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 12:42:43.461375 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.461290 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 12:42:43.461604 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.461584 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.462321 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.462296 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tpgh5\"" Jul 07 12:42:43.462432 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.462296 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.462786 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.462767 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.464062 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.463994 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 12:42:43.470215 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.469920 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.470388 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.470291 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ctvp4\"" Jul 07 12:42:43.470461 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.470446 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 12:42:43.470461 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.470457 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.470609 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.470553 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 12:42:43.471846 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.471098 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8"] Jul 07 12:42:43.472457 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.472434 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-d67d7"] Jul 07 12:42:43.475798 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.475776 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ckz8l"] Jul 07 12:42:43.475937 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.475919 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.478712 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.478685 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.478863 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.478841 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rgl4g\"" Jul 07 12:42:43.479006 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.478957 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 12:42:43.479135 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.478744 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 12:42:43.479658 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.479637 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.480172 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.480029 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk"] Jul 07 12:42:43.480172 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.480153 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.482521 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.482224 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9"] Jul 07 12:42:43.482619 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.482556 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:43.483940 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.483203 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.488704 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.487231 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8"] Jul 07 12:42:43.488704 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.487257 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.488704 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.487277 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll"] Jul 07 12:42:43.488704 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.487303 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cd865c99-qfscz"] Jul 07 12:42:43.488704 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.487333 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4xbtp"] Jul 07 12:42:43.488704 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.487771 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 12:42:43.493302 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.493281 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 12:42:43.493725 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.493703 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 12:42:43.494035 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.494014 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.494132 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.494085 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.494304 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.493724 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-67ff897d47-8lqlg"] Jul 07 12:42:43.495956 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.495935 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.499065 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.496399 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 12:42:43.499065 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.496833 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nfxgv\"" Jul 07 12:42:43.499065 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.496999 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-z9p49\"" Jul 07 12:42:43.499065 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.497442 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.499065 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.498620 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.499065 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.498923 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ncl7q\"" Jul 07 12:42:43.499437 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.499397 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-z24dh\"" Jul 07 12:42:43.500562 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.500542 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 12:42:43.502424 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.502406 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-67776"] Jul 07 12:42:43.503286 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.502599 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.504319 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.504300 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 12:42:43.504415 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.504382 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 12:42:43.504478 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.504438 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.504612 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.504596 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 12:42:43.504766 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.504730 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 12:42:43.505203 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.505182 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-26jzv"] Jul 07 12:42:43.505347 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.505331 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:43.506313 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.506297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 12:42:43.506482 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.506463 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.506781 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.506761 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 12:42:43.507024 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.507006 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xdn77\"" Jul 07 12:42:43.507300 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.507282 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 12:42:43.507382 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.507341 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 12:42:43.507640 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.507619 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.507944 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.507926 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 12:42:43.508195 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.508176 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 12:42:43.508384 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.508359 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6x89k\"" Jul 07 12:42:43.508898 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.508879 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq"] Jul 07 12:42:43.509173 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.509097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.510852 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510833 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ckz8l"] Jul 07 12:42:43.510927 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510870 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk"] Jul 07 12:42:43.510927 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510896 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-d67d7"] Jul 07 12:42:43.510927 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510915 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9"] Jul 07 12:42:43.511055 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510938 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-67776"] Jul 07 12:42:43.511055 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510958 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq"] Jul 07 12:42:43.511055 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510976 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-26jzv"] Jul 07 12:42:43.511055 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.510994 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67ff897d47-8lqlg"] Jul 07 12:42:43.511055 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.511013 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4xbtp"] Jul 07 12:42:43.511279 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.511124 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" Jul 07 12:42:43.511686 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.511668 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 12:42:43.511943 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.511924 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.515116 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.515096 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zphhp\"" Jul 07 12:42:43.515209 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.515147 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 12:42:43.515662 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.515298 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.515662 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.515403 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 12:42:43.515662 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.515548 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mbsnk\"" Jul 07 12:42:43.600577 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k97d5\" (UniqueName: \"kubernetes.io/projected/53e4e2b2-0715-449f-b683-43191049e45a-kube-api-access-k97d5\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.600752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqnfc\" (UniqueName: \"kubernetes.io/projected/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-kube-api-access-hqnfc\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.600752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2e4302f-b012-41ab-afc0-28c1f36f5389-config-volume\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.600752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2e4302f-b012-41ab-afc0-28c1f36f5389-tmp-dir\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.600752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.600752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nccc\" (UniqueName: \"kubernetes.io/projected/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-kube-api-access-6nccc\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.600752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600713 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:43.600752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600743 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzmr\" (UniqueName: \"kubernetes.io/projected/f2486c20-e7ba-45f6-8275-8f3c6138037d-kube-api-access-btzmr\") pod \"network-check-source-989f8b8dc-d4gvq\" (UID: \"f2486c20-e7ba-45f6-8275-8f3c6138037d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" Jul 07 12:42:43.601067 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53e4e2b2-0715-449f-b683-43191049e45a-serving-cert\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.601067 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.601067 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09307f15-ddb7-4d32-9af7-6021f032283e-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.601067 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87lxv\" (UniqueName: \"kubernetes.io/projected/d2e4302f-b012-41ab-afc0-28c1f36f5389-kube-api-access-87lxv\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.601067 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.600966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khm4g\" (UniqueName: \"kubernetes.io/projected/d3580b95-a6d3-4838-bac6-5c65e87f1760-kube-api-access-khm4g\") pod \"volume-data-source-validator-5d456fdbcd-nlgs8\" (UID: \"d3580b95-a6d3-4838-bac6-5c65e87f1760\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" Jul 07 12:42:43.601067 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:43.601067 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-image-registry-private-configuration\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-trusted-ca\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0430f99a-b20d-4fb2-b994-1d150ef9cb30-serving-cert\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601125 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53e4e2b2-0715-449f-b683-43191049e45a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2f6z\" (UniqueName: \"kubernetes.io/projected/813dbf62-2e9f-458d-93aa-693b6c4bb673-kube-api-access-b2f6z\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-stats-auth\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtznx\" (UniqueName: \"kubernetes.io/projected/0430f99a-b20d-4fb2-b994-1d150ef9cb30-kube-api-access-gtznx\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601210 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhh6x\" (UniqueName: \"kubernetes.io/projected/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-kube-api-access-lhh6x\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0430f99a-b20d-4fb2-b994-1d150ef9cb30-config\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c25b375a-dac9-478a-b4ec-9a75fe8fd446-ca-trust-extracted\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.601297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601298 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/53e4e2b2-0715-449f-b683-43191049e45a-snapshots\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601313 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-installation-pull-secrets\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-config\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601369 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1c22b51-fd94-4663-86d4-53c79d11996e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcxq9\" (UniqueName: \"kubernetes.io/projected/05a7a66a-0fe9-4c97-a899-d5c914212263-kube-api-access-kcxq9\") pod \"ingress-canary-26jzv\" (UID: \"05a7a66a-0fe9-4c97-a899-d5c914212263\") " pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09307f15-ddb7-4d32-9af7-6021f032283e-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkzsz\" (UniqueName: \"kubernetes.io/projected/09307f15-ddb7-4d32-9af7-6021f032283e-kube-api-access-lkzsz\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sz88\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-kube-api-access-5sz88\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601560 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601582 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601606 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05a7a66a-0fe9-4c97-a899-d5c914212263-cert\") pod \"ingress-canary-26jzv\" (UID: \"05a7a66a-0fe9-4c97-a899-d5c914212263\") " pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0430f99a-b20d-4fb2-b994-1d150ef9cb30-trusted-ca\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53e4e2b2-0715-449f-b683-43191049e45a-tmp\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.601793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601711 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-default-certificate\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.602259 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.602259 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601766 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53e4e2b2-0715-449f-b683-43191049e45a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.602259 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-certificates\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.602259 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.601799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-bound-sa-token\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.703068 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.702966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/53e4e2b2-0715-449f-b683-43191049e45a-snapshots\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.703068 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.703068 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-installation-pull-secrets\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.703343 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-config\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.703343 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.703142 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 12:42:43.703343 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.703225 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:44.203202097 +0000 UTC m=+37.547856278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : secret "router-metrics-certs-default" not found Jul 07 12:42:43.703343 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1c22b51-fd94-4663-86d4-53c79d11996e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:43.703343 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcxq9\" (UniqueName: \"kubernetes.io/projected/05a7a66a-0fe9-4c97-a899-d5c914212263-kube-api-access-kcxq9\") pod \"ingress-canary-26jzv\" (UID: \"05a7a66a-0fe9-4c97-a899-d5c914212263\") " pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.703343 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09307f15-ddb7-4d32-9af7-6021f032283e-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.703343 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkzsz\" (UniqueName: \"kubernetes.io/projected/09307f15-ddb7-4d32-9af7-6021f032283e-kube-api-access-lkzsz\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sz88\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-kube-api-access-5sz88\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05a7a66a-0fe9-4c97-a899-d5c914212263-cert\") pod \"ingress-canary-26jzv\" (UID: \"05a7a66a-0fe9-4c97-a899-d5c914212263\") " pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0430f99a-b20d-4fb2-b994-1d150ef9cb30-trusted-ca\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53e4e2b2-0715-449f-b683-43191049e45a-tmp\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-default-certificate\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53e4e2b2-0715-449f-b683-43191049e45a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-certificates\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-bound-sa-token\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703625 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-config\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k97d5\" (UniqueName: \"kubernetes.io/projected/53e4e2b2-0715-449f-b683-43191049e45a-kube-api-access-k97d5\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqnfc\" (UniqueName: \"kubernetes.io/projected/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-kube-api-access-hqnfc\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.703701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/53e4e2b2-0715-449f-b683-43191049e45a-snapshots\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2e4302f-b012-41ab-afc0-28c1f36f5389-config-volume\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2e4302f-b012-41ab-afc0-28c1f36f5389-tmp-dir\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nccc\" (UniqueName: \"kubernetes.io/projected/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-kube-api-access-6nccc\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.703795 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.703854 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls podName:1acc0eb6-6234-43f8-ae0e-fda54a04ce53 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:44.203837633 +0000 UTC m=+37.548491809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6llv9" (UID: "1acc0eb6-6234-43f8-ae0e-fda54a04ce53") : secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btzmr\" (UniqueName: \"kubernetes.io/projected/f2486c20-e7ba-45f6-8275-8f3c6138037d-kube-api-access-btzmr\") pod \"network-check-source-989f8b8dc-d4gvq\" (UID: \"f2486c20-e7ba-45f6-8275-8f3c6138037d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.703923 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.703968 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert podName:f1c22b51-fd94-4663-86d4-53c79d11996e nodeName:}" failed. No retries permitted until 2026-07-07 12:42:44.203954072 +0000 UTC m=+37.548608262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-67776" (UID: "f1c22b51-fd94-4663-86d4-53c79d11996e") : secret "networking-console-plugin-cert" not found Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53e4e2b2-0715-449f-b683-43191049e45a-serving-cert\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.703996 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09307f15-ddb7-4d32-9af7-6021f032283e-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87lxv\" (UniqueName: \"kubernetes.io/projected/d2e4302f-b012-41ab-afc0-28c1f36f5389-kube-api-access-87lxv\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.704549 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khm4g\" (UniqueName: \"kubernetes.io/projected/d3580b95-a6d3-4838-bac6-5c65e87f1760-kube-api-access-khm4g\") pod \"volume-data-source-validator-5d456fdbcd-nlgs8\" (UID: \"d3580b95-a6d3-4838-bac6-5c65e87f1760\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-image-registry-private-configuration\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704173 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-trusted-ca\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0430f99a-b20d-4fb2-b994-1d150ef9cb30-serving-cert\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53e4e2b2-0715-449f-b683-43191049e45a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704313 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f1c22b51-fd94-4663-86d4-53c79d11996e-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704636 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53e4e2b2-0715-449f-b683-43191049e45a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.703998 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.704673 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd865c99-qfscz: secret "image-registry-tls" not found Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.704658 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:44.204642319 +0000 UTC m=+37.549296494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : configmap references non-existent config key: service-ca.crt Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.704706 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls podName:c25b375a-dac9-478a-b4ec-9a75fe8fd446 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:44.204697579 +0000 UTC m=+37.549351741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls") pod "image-registry-5cd865c99-qfscz" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446") : secret "image-registry-tls" not found Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.704908 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-certificates\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.705287 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.705264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.705910 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.705743 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0430f99a-b20d-4fb2-b994-1d150ef9cb30-trusted-ca\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.705910 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.705760 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09307f15-ddb7-4d32-9af7-6021f032283e-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.706071 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.706033 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53e4e2b2-0715-449f-b683-43191049e45a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.706326 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.706305 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 12:42:43.706410 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.706363 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls podName:813dbf62-2e9f-458d-93aa-693b6c4bb673 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:44.206348718 +0000 UTC m=+37.551002896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5hnzk" (UID: "813dbf62-2e9f-458d-93aa-693b6c4bb673") : secret "samples-operator-tls" not found Jul 07 12:42:43.706554 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.706533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2f6z\" (UniqueName: \"kubernetes.io/projected/813dbf62-2e9f-458d-93aa-693b6c4bb673-kube-api-access-b2f6z\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:43.706690 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.706669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-stats-auth\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.706835 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.706805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtznx\" (UniqueName: \"kubernetes.io/projected/0430f99a-b20d-4fb2-b994-1d150ef9cb30-kube-api-access-gtznx\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.706994 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.706973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhh6x\" (UniqueName: \"kubernetes.io/projected/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-kube-api-access-lhh6x\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.707182 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.707160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0430f99a-b20d-4fb2-b994-1d150ef9cb30-config\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.707732 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.707709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c25b375a-dac9-478a-b4ec-9a75fe8fd446-ca-trust-extracted\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.707902 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.707332 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2e4302f-b012-41ab-afc0-28c1f36f5389-tmp-dir\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.708010 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.706855 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2e4302f-b012-41ab-afc0-28c1f36f5389-config-volume\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.708153 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.708005 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53e4e2b2-0715-449f-b683-43191049e45a-tmp\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.708246 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.707298 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-trusted-ca\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.708335 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.706564 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 12:42:43.708476 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:43.708463 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls podName:d2e4302f-b012-41ab-afc0-28c1f36f5389 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:44.208444079 +0000 UTC m=+37.553098253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls") pod "dns-default-4xbtp" (UID: "d2e4302f-b012-41ab-afc0-28c1f36f5389") : secret "dns-default-metrics-tls" not found Jul 07 12:42:43.708570 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.708375 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0430f99a-b20d-4fb2-b994-1d150ef9cb30-config\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.708665 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.708348 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c25b375a-dac9-478a-b4ec-9a75fe8fd446-ca-trust-extracted\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.708752 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.708654 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09307f15-ddb7-4d32-9af7-6021f032283e-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.708836 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.708653 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.709753 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.709721 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-image-registry-private-configuration\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.709838 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.709811 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0430f99a-b20d-4fb2-b994-1d150ef9cb30-serving-cert\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.710651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.710628 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-installation-pull-secrets\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.710746 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.710681 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53e4e2b2-0715-449f-b683-43191049e45a-serving-cert\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.710746 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.710744 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-default-certificate\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.711112 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.711090 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-stats-auth\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.711805 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.711785 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/05a7a66a-0fe9-4c97-a899-d5c914212263-cert\") pod \"ingress-canary-26jzv\" (UID: \"05a7a66a-0fe9-4c97-a899-d5c914212263\") " pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.714148 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.714125 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87lxv\" (UniqueName: \"kubernetes.io/projected/d2e4302f-b012-41ab-afc0-28c1f36f5389-kube-api-access-87lxv\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:43.714722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.714698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k97d5\" (UniqueName: \"kubernetes.io/projected/53e4e2b2-0715-449f-b683-43191049e45a-kube-api-access-k97d5\") pod \"insights-operator-5c847dcfc4-ckz8l\" (UID: \"53e4e2b2-0715-449f-b683-43191049e45a\") " pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.714993 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.714967 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nccc\" (UniqueName: \"kubernetes.io/projected/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-kube-api-access-6nccc\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:43.715398 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.715373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sz88\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-kube-api-access-5sz88\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.717218 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.717195 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-bound-sa-token\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:43.717376 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.717353 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2f6z\" (UniqueName: \"kubernetes.io/projected/813dbf62-2e9f-458d-93aa-693b6c4bb673-kube-api-access-b2f6z\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:43.717642 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.717624 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzmr\" (UniqueName: \"kubernetes.io/projected/f2486c20-e7ba-45f6-8275-8f3c6138037d-kube-api-access-btzmr\") pod \"network-check-source-989f8b8dc-d4gvq\" (UID: \"f2486c20-e7ba-45f6-8275-8f3c6138037d\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" Jul 07 12:42:43.717715 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.717666 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqnfc\" (UniqueName: \"kubernetes.io/projected/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-kube-api-access-hqnfc\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:43.717769 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.717711 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcxq9\" (UniqueName: \"kubernetes.io/projected/05a7a66a-0fe9-4c97-a899-d5c914212263-kube-api-access-kcxq9\") pod \"ingress-canary-26jzv\" (UID: \"05a7a66a-0fe9-4c97-a899-d5c914212263\") " pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.717769 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.717760 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkzsz\" (UniqueName: \"kubernetes.io/projected/09307f15-ddb7-4d32-9af7-6021f032283e-kube-api-access-lkzsz\") pod \"kube-storage-version-migrator-operator-846bf668f7-dkkf8\" (UID: \"09307f15-ddb7-4d32-9af7-6021f032283e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.718086 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.718063 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khm4g\" (UniqueName: \"kubernetes.io/projected/d3580b95-a6d3-4838-bac6-5c65e87f1760-kube-api-access-khm4g\") pod \"volume-data-source-validator-5d456fdbcd-nlgs8\" (UID: \"d3580b95-a6d3-4838-bac6-5c65e87f1760\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" Jul 07 12:42:43.719889 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.719867 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhh6x\" (UniqueName: \"kubernetes.io/projected/08c14ad2-d6d6-4f75-a63b-6db8b8714b39-kube-api-access-lhh6x\") pod \"service-ca-operator-5d46d9ffd4-c6fll\" (UID: \"08c14ad2-d6d6-4f75-a63b-6db8b8714b39\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.723594 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.723574 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtznx\" (UniqueName: \"kubernetes.io/projected/0430f99a-b20d-4fb2-b994-1d150ef9cb30-kube-api-access-gtznx\") pod \"console-operator-68b757fb67-d67d7\" (UID: \"0430f99a-b20d-4fb2-b994-1d150ef9cb30\") " pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.771088 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.771025 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" Jul 07 12:42:43.794234 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.794203 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" Jul 07 12:42:43.808957 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.808931 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" Jul 07 12:42:43.824163 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.824130 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:42:43.830407 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.830331 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" Jul 07 12:42:43.866993 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.866683 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-26jzv" Jul 07 12:42:43.872942 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:43.872854 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" Jul 07 12:42:44.021402 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.018751 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8"] Jul 07 12:42:44.021402 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:44.021305 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09307f15_ddb7_4d32_9af7_6021f032283e.slice/crio-3ae91d65be1684c16177bafafb69524d967640b9ac4307bc4262a67eb5d15e39 WatchSource:0}: Error finding container 3ae91d65be1684c16177bafafb69524d967640b9ac4307bc4262a67eb5d15e39: Status 404 returned error can't find the container with id 3ae91d65be1684c16177bafafb69524d967640b9ac4307bc4262a67eb5d15e39 Jul 07 12:42:44.058520 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.057741 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll"] Jul 07 12:42:44.064575 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:44.063781 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c14ad2_d6d6_4f75_a63b_6db8b8714b39.slice/crio-ea8137053daa84b95983d25005d92f0a66df9c14b14484f2f4917eb693059b19 WatchSource:0}: Error finding container ea8137053daa84b95983d25005d92f0a66df9c14b14484f2f4917eb693059b19: Status 404 returned error can't find the container with id ea8137053daa84b95983d25005d92f0a66df9c14b14484f2f4917eb693059b19 Jul 07 12:42:44.067699 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.067675 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8"] Jul 07 12:42:44.075242 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:44.074972 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3580b95_a6d3_4838_bac6_5c65e87f1760.slice/crio-658e7baf9ebf797241ad62359ce93e13589138439975ea246936790e69fdb874 WatchSource:0}: Error finding container 658e7baf9ebf797241ad62359ce93e13589138439975ea246936790e69fdb874: Status 404 returned error can't find the container with id 658e7baf9ebf797241ad62359ce93e13589138439975ea246936790e69fdb874 Jul 07 12:42:44.098170 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.098095 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ckz8l"] Jul 07 12:42:44.100669 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.100235 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-d67d7"] Jul 07 12:42:44.101573 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:44.101470 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e4e2b2_0715_449f_b683_43191049e45a.slice/crio-ce7f222241be241831897280291137509d7cc04cf0e8c43a54971d3b5eae2b3b WatchSource:0}: Error finding container ce7f222241be241831897280291137509d7cc04cf0e8c43a54971d3b5eae2b3b: Status 404 returned error can't find the container with id ce7f222241be241831897280291137509d7cc04cf0e8c43a54971d3b5eae2b3b Jul 07 12:42:44.118718 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.118683 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-26jzv"] Jul 07 12:42:44.122008 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:44.121975 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a7a66a_0fe9_4c97_a899_d5c914212263.slice/crio-949eac886d3d2626d13873b1153a2f813565ea921ea625f08749ff730c7e51aa WatchSource:0}: Error finding container 949eac886d3d2626d13873b1153a2f813565ea921ea625f08749ff730c7e51aa: Status 404 returned error can't find the container with id 949eac886d3d2626d13873b1153a2f813565ea921ea625f08749ff730c7e51aa Jul 07 12:42:44.151282 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.151254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:42:44.151687 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.151645 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:42:44.151811 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.151645 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:42:44.151940 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.151916 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq"] Jul 07 12:42:44.154262 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.154240 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 12:42:44.154369 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.154284 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 12:42:44.154369 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.154297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-849mb\"" Jul 07 12:42:44.154477 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.154244 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gxjtk\"" Jul 07 12:42:44.156492 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:42:44.156461 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2486c20_e7ba_45f6_8275_8f3c6138037d.slice/crio-a62ffba655f917929e1f30f7c985f03bf25687688bda1dbcb20511e2f8278039 WatchSource:0}: Error finding container a62ffba655f917929e1f30f7c985f03bf25687688bda1dbcb20511e2f8278039: Status 404 returned error can't find the container with id a62ffba655f917929e1f30f7c985f03bf25687688bda1dbcb20511e2f8278039 Jul 07 12:42:44.215353 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.215229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:44.215353 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.215287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:44.215353 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.215331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215391 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215411 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd865c99-qfscz: secret "image-registry-tls" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.215394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215454 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215497 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215456 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215470 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls podName:c25b375a-dac9-478a-b4ec-9a75fe8fd446 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.215451789 +0000 UTC m=+38.560105971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls") pod "image-registry-5cd865c99-qfscz" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446") : secret "image-registry-tls" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215588 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls podName:d2e4302f-b012-41ab-afc0-28c1f36f5389 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.215571191 +0000 UTC m=+38.560225353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls") pod "dns-default-4xbtp" (UID: "d2e4302f-b012-41ab-afc0-28c1f36f5389") : secret "dns-default-metrics-tls" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215619 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls podName:1acc0eb6-6234-43f8-ae0e-fda54a04ce53 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.21560228 +0000 UTC m=+38.560256455 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6llv9" (UID: "1acc0eb6-6234-43f8-ae0e-fda54a04ce53") : secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:44.215638 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215637 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert podName:f1c22b51-fd94-4663-86d4-53c79d11996e nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.215630374 +0000 UTC m=+38.560284537 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-67776" (UID: "f1c22b51-fd94-4663-86d4-53c79d11996e") : secret "networking-console-plugin-cert" not found Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.215687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.215711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.215776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215817 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.215802613 +0000 UTC m=+38.560456792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : configmap references non-existent config key: service-ca.crt Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215869 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215890 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215906 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.215894442 +0000 UTC m=+38.560548620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : secret "router-metrics-certs-default" not found Jul 07 12:42:44.216211 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:44.215957 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls podName:813dbf62-2e9f-458d-93aa-693b6c4bb673 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:45.215944045 +0000 UTC m=+38.560598222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5hnzk" (UID: "813dbf62-2e9f-458d-93aa-693b6c4bb673") : secret "samples-operator-tls" not found Jul 07 12:42:44.363718 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.363675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-26jzv" event={"ID":"05a7a66a-0fe9-4c97-a899-d5c914212263","Type":"ContainerStarted","Data":"949eac886d3d2626d13873b1153a2f813565ea921ea625f08749ff730c7e51aa"} Jul 07 12:42:44.365106 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.365068 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" event={"ID":"09307f15-ddb7-4d32-9af7-6021f032283e","Type":"ContainerStarted","Data":"3ae91d65be1684c16177bafafb69524d967640b9ac4307bc4262a67eb5d15e39"} Jul 07 12:42:44.366617 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.366583 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" event={"ID":"f2486c20-e7ba-45f6-8275-8f3c6138037d","Type":"ContainerStarted","Data":"a62ffba655f917929e1f30f7c985f03bf25687688bda1dbcb20511e2f8278039"} Jul 07 12:42:44.367756 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.367717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" event={"ID":"53e4e2b2-0715-449f-b683-43191049e45a","Type":"ContainerStarted","Data":"ce7f222241be241831897280291137509d7cc04cf0e8c43a54971d3b5eae2b3b"} Jul 07 12:42:44.369246 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.369202 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" event={"ID":"0430f99a-b20d-4fb2-b994-1d150ef9cb30","Type":"ContainerStarted","Data":"fc0bc2ae6093cad2d2b8317edcb1a8e572ac0a973d88eda93da24af8f73931fb"} Jul 07 12:42:44.370461 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.370434 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" event={"ID":"08c14ad2-d6d6-4f75-a63b-6db8b8714b39","Type":"ContainerStarted","Data":"ea8137053daa84b95983d25005d92f0a66df9c14b14484f2f4917eb693059b19"} Jul 07 12:42:44.372367 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:44.372339 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" event={"ID":"d3580b95-a6d3-4838-bac6-5c65e87f1760","Type":"ContainerStarted","Data":"658e7baf9ebf797241ad62359ce93e13589138439975ea246936790e69fdb874"} Jul 07 12:42:45.225397 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:45.225356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:45.225441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:45.225476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:45.225537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:45.225569 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:45.225602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:45.225641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.225805 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:45.226237 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.225870 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls podName:1acc0eb6-6234-43f8-ae0e-fda54a04ce53 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:47.225849763 +0000 UTC m=+40.570503940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6llv9" (UID: "1acc0eb6-6234-43f8-ae0e-fda54a04ce53") : secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226293 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226343 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert podName:f1c22b51-fd94-4663-86d4-53c79d11996e nodeName:}" failed. No retries permitted until 2026-07-07 12:42:47.226329092 +0000 UTC m=+40.570983268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-67776" (UID: "f1c22b51-fd94-4663-86d4-53c79d11996e") : secret "networking-console-plugin-cert" not found Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226414 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:47.226401935 +0000 UTC m=+40.571056114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : configmap references non-existent config key: service-ca.crt Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226470 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226500 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls podName:813dbf62-2e9f-458d-93aa-693b6c4bb673 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:47.226489856 +0000 UTC m=+40.571144032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5hnzk" (UID: "813dbf62-2e9f-458d-93aa-693b6c4bb673") : secret "samples-operator-tls" not found Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226554 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226582 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:47.226571366 +0000 UTC m=+40.571225542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : secret "router-metrics-certs-default" not found Jul 07 12:42:45.226645 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226641 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 12:42:45.227053 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226653 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd865c99-qfscz: secret "image-registry-tls" not found Jul 07 12:42:45.227053 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226683 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls podName:c25b375a-dac9-478a-b4ec-9a75fe8fd446 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:47.226673508 +0000 UTC m=+40.571327675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls") pod "image-registry-5cd865c99-qfscz" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446") : secret "image-registry-tls" not found Jul 07 12:42:45.227053 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226737 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 12:42:45.227053 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:45.226763 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls podName:d2e4302f-b012-41ab-afc0-28c1f36f5389 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:47.226754569 +0000 UTC m=+40.571408734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls") pod "dns-default-4xbtp" (UID: "d2e4302f-b012-41ab-afc0-28c1f36f5389") : secret "dns-default-metrics-tls" not found Jul 07 12:42:47.248251 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:47.248205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:47.248261 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:47.248296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:47.248336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248385 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:47.248401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248414 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248429 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd865c99-qfscz: secret "image-registry-tls" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:47.248452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248466 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.248445654 +0000 UTC m=+44.593099836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : secret "router-metrics-certs-default" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248465 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248511 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248513 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls podName:c25b375a-dac9-478a-b4ec-9a75fe8fd446 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.248503892 +0000 UTC m=+44.593158054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls") pod "image-registry-5cd865c99-qfscz" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446") : secret "image-registry-tls" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248525 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248532 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls podName:d2e4302f-b012-41ab-afc0-28c1f36f5389 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.248524085 +0000 UTC m=+44.593178251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls") pod "dns-default-4xbtp" (UID: "d2e4302f-b012-41ab-afc0-28c1f36f5389") : secret "dns-default-metrics-tls" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248600 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert podName:f1c22b51-fd94-4663-86d4-53c79d11996e nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.24858025 +0000 UTC m=+44.593234427 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-67776" (UID: "f1c22b51-fd94-4663-86d4-53c79d11996e") : secret "networking-console-plugin-cert" not found Jul 07 12:42:47.248710 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248619 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.248609583 +0000 UTC m=+44.593263746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : configmap references non-existent config key: service-ca.crt Jul 07 12:42:47.249272 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248636 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls podName:1acc0eb6-6234-43f8-ae0e-fda54a04ce53 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.24862714 +0000 UTC m=+44.593281301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6llv9" (UID: "1acc0eb6-6234-43f8-ae0e-fda54a04ce53") : secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:47.249272 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:47.248669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:47.249272 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248770 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 12:42:47.249272 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:47.248826 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls podName:813dbf62-2e9f-458d-93aa-693b6c4bb673 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:51.248801554 +0000 UTC m=+44.593455715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5hnzk" (UID: "813dbf62-2e9f-458d-93aa-693b6c4bb673") : secret "samples-operator-tls" not found Jul 07 12:42:51.288916 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:51.288877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:51.288974 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:51.289029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289052 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:51.289089 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289115 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289141 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls podName:1acc0eb6-6234-43f8-ae0e-fda54a04ce53 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.289121195 +0000 UTC m=+52.633775357 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6llv9" (UID: "1acc0eb6-6234-43f8-ae0e-fda54a04ce53") : secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:51.289200 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289237 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert podName:f1c22b51-fd94-4663-86d4-53c79d11996e nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.289214433 +0000 UTC m=+52.633868611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-67776" (UID: "f1c22b51-fd94-4663-86d4-53c79d11996e") : secret "networking-console-plugin-cert" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289250 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289272 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.289259448 +0000 UTC m=+52.633913614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : configmap references non-existent config key: service-ca.crt Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289277 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289310 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls podName:813dbf62-2e9f-458d-93aa-693b6c4bb673 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.289297004 +0000 UTC m=+52.633951166 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5hnzk" (UID: "813dbf62-2e9f-458d-93aa-693b6c4bb673") : secret "samples-operator-tls" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289325 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.289319254 +0000 UTC m=+52.633973415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : secret "router-metrics-certs-default" not found Jul 07 12:42:51.289456 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:51.289341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:51.289902 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:51.289378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:51.289902 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289431 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 12:42:51.289902 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289442 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd865c99-qfscz: secret "image-registry-tls" not found Jul 07 12:42:51.289902 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289458 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 12:42:51.289902 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289484 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls podName:c25b375a-dac9-478a-b4ec-9a75fe8fd446 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.28947035 +0000 UTC m=+52.634124526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls") pod "image-registry-5cd865c99-qfscz" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446") : secret "image-registry-tls" not found Jul 07 12:42:51.289902 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:51.289499 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls podName:d2e4302f-b012-41ab-afc0-28c1f36f5389 nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.289491573 +0000 UTC m=+52.634145736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls") pod "dns-default-4xbtp" (UID: "d2e4302f-b012-41ab-afc0-28c1f36f5389") : secret "dns-default-metrics-tls" not found Jul 07 12:42:54.411293 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.411219 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerStarted","Data":"ba7edc825e18b27aefb8bb0801767cf0d320a0c053bcd185df66d959379ceec8"} Jul 07 12:42:54.413350 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.413321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" event={"ID":"53e4e2b2-0715-449f-b683-43191049e45a","Type":"ContainerStarted","Data":"036137661a4dc7cb38d3e9b1c57b96633c174505f50a16d3741f8b4428acc661"} Jul 07 12:42:54.417071 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.416076 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/0.log" Jul 07 12:42:54.417071 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.416129 2578 generic.go:356] "Generic (PLEG): container finished" podID="0430f99a-b20d-4fb2-b994-1d150ef9cb30" containerID="f6326c1cfb45e54cce4ab5288011c2d9926a268e7aeb9bd74f0f52bd7e77a136" exitCode=255 Jul 07 12:42:54.417071 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.416568 2578 scope.go:117] "RemoveContainer" containerID="f6326c1cfb45e54cce4ab5288011c2d9926a268e7aeb9bd74f0f52bd7e77a136" Jul 07 12:42:54.417071 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.416816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" event={"ID":"0430f99a-b20d-4fb2-b994-1d150ef9cb30","Type":"ContainerDied","Data":"f6326c1cfb45e54cce4ab5288011c2d9926a268e7aeb9bd74f0f52bd7e77a136"} Jul 07 12:42:54.420024 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.419994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" event={"ID":"08c14ad2-d6d6-4f75-a63b-6db8b8714b39","Type":"ContainerStarted","Data":"285676327c97667a42f57c213cbbacf312de153f88747ed9853803cf80b62676"} Jul 07 12:42:54.423012 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.422987 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" event={"ID":"d3580b95-a6d3-4838-bac6-5c65e87f1760","Type":"ContainerStarted","Data":"24c191d73d42ca0be9f4f4e41ddd72e7f57bb4d9585ee068b519d3dadae72a54"} Jul 07 12:42:54.424768 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.424744 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-26jzv" event={"ID":"05a7a66a-0fe9-4c97-a899-d5c914212263","Type":"ContainerStarted","Data":"b904f44eaf56a7a4c165fd2f5bdf969efbf8e85992682b4020a6db044a02f600"} Jul 07 12:42:54.426541 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.426515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" event={"ID":"09307f15-ddb7-4d32-9af7-6021f032283e","Type":"ContainerStarted","Data":"d5f5a002dfa617c9c8a10c51ecf87b31577afbcfbc8ff64cf4df41cccf5d1324"} Jul 07 12:42:54.428001 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.427973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" event={"ID":"f2486c20-e7ba-45f6-8275-8f3c6138037d","Type":"ContainerStarted","Data":"2bf2ea555b709f91bc81cfeca455ea1a53bee51e2ae2c95cf614aead7ff01462"} Jul 07 12:42:54.513480 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.513333 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" podStartSLOduration=7.7223382229999995 podStartE2EDuration="17.513307115s" totalImagesPullingTime="9.790968892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:54.477236277 +0000 UTC m=+47.821890463" watchObservedRunningTime="2026-07-07 12:42:54.513307115 +0000 UTC m=+47.857961295" Jul 07 12:42:54.541124 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.539492 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" podStartSLOduration=7.786162871 podStartE2EDuration="17.539469254s" totalImagesPullingTime="9.753306383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:54.506352744 +0000 UTC m=+47.851006927" watchObservedRunningTime="2026-07-07 12:42:54.539469254 +0000 UTC m=+47.884123439" Jul 07 12:42:54.572512 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.572465 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" podStartSLOduration=7.745663302 podStartE2EDuration="17.572441831s" totalImagesPullingTime="9.826778529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:54.550460499 +0000 UTC m=+47.895114684" watchObservedRunningTime="2026-07-07 12:42:54.572441831 +0000 UTC m=+47.917096015" Jul 07 12:42:54.573013 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.572985 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-26jzv" podStartSLOduration=1.8324967650000001 podStartE2EDuration="11.572974391s" totalImagesPullingTime="9.740477626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:54.57044322 +0000 UTC m=+47.915097429" watchObservedRunningTime="2026-07-07 12:42:54.572974391 +0000 UTC m=+47.917628576" Jul 07 12:42:54.588366 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.588285 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-nlgs8" podStartSLOduration=7.885348989 podStartE2EDuration="17.588263248s" totalImagesPullingTime="9.702914259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:54.587166588 +0000 UTC m=+47.931820775" watchObservedRunningTime="2026-07-07 12:42:54.588263248 +0000 UTC m=+47.932917434" Jul 07 12:42:54.607548 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:54.607387 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-d4gvq" podStartSLOduration=7.658402065 podStartE2EDuration="17.607364048s" totalImagesPullingTime="9.948961983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:54.606780305 +0000 UTC m=+47.951434492" watchObservedRunningTime="2026-07-07 12:42:54.607364048 +0000 UTC m=+47.952018233" Jul 07 12:42:55.436977 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.436879 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7392278-af8b-4f18-b906-0632aeda6a07" containerID="ba7edc825e18b27aefb8bb0801767cf0d320a0c053bcd185df66d959379ceec8" exitCode=0 Jul 07 12:42:55.436977 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.436910 2578 generic.go:356] "Generic (PLEG): container finished" podID="b7392278-af8b-4f18-b906-0632aeda6a07" containerID="0d307282e3fb47b02b607b31a8ec8292b921045cf8159e6cc430f9f61d954a7c" exitCode=0 Jul 07 12:42:55.437468 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.436967 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerDied","Data":"ba7edc825e18b27aefb8bb0801767cf0d320a0c053bcd185df66d959379ceec8"} Jul 07 12:42:55.437468 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.437025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerDied","Data":"0d307282e3fb47b02b607b31a8ec8292b921045cf8159e6cc430f9f61d954a7c"} Jul 07 12:42:55.439797 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.439772 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:42:55.440302 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.440272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/0.log" Jul 07 12:42:55.440409 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.440311 2578 generic.go:356] "Generic (PLEG): container finished" podID="0430f99a-b20d-4fb2-b994-1d150ef9cb30" containerID="6a8634670d304b369eaa16d470acd6782c520a02790edd1099d4cd0c25a437d3" exitCode=255 Jul 07 12:42:55.440642 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.440629 2578 scope.go:117] "RemoveContainer" containerID="6a8634670d304b369eaa16d470acd6782c520a02790edd1099d4cd0c25a437d3" Jul 07 12:42:55.440806 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:55.440786 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-d67d7_openshift-console-operator(0430f99a-b20d-4fb2-b994-1d150ef9cb30)\"" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" podUID="0430f99a-b20d-4fb2-b994-1d150ef9cb30" Jul 07 12:42:55.440849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.440828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" event={"ID":"0430f99a-b20d-4fb2-b994-1d150ef9cb30","Type":"ContainerDied","Data":"6a8634670d304b369eaa16d470acd6782c520a02790edd1099d4cd0c25a437d3"} Jul 07 12:42:55.440886 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:55.440868 2578 scope.go:117] "RemoveContainer" containerID="f6326c1cfb45e54cce4ab5288011c2d9926a268e7aeb9bd74f0f52bd7e77a136" Jul 07 12:42:56.161290 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.161249 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ht647"] Jul 07 12:42:56.187180 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.187146 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ht647"] Jul 07 12:42:56.187336 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.187256 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.189801 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.189762 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 12:42:56.189941 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.189826 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 12:42:56.189941 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.189884 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdzl4\"" Jul 07 12:42:56.243457 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.243420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96a07f6-2590-4338-8df3-4cfe79d22b4c-crio-socket\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.243651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.243464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgqq\" (UniqueName: \"kubernetes.io/projected/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-api-access-slgqq\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.243651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.243611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96a07f6-2590-4338-8df3-4cfe79d22b4c-data-volume\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.243763 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.243649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.243763 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.243714 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.344682 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.344643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.344848 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.344751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96a07f6-2590-4338-8df3-4cfe79d22b4c-crio-socket\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.344848 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.344787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgqq\" (UniqueName: \"kubernetes.io/projected/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-api-access-slgqq\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.344848 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.344810 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 12:42:56.345021 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.344880 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:42:56.844858638 +0000 UTC m=+50.189512823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : secret "insights-runtime-extractor-tls" not found Jul 07 12:42:56.345021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.344938 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96a07f6-2590-4338-8df3-4cfe79d22b4c-crio-socket\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.345164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.345064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96a07f6-2590-4338-8df3-4cfe79d22b4c-data-volume\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.345164 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.345093 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.345272 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.345187 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 12:42:56.345272 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.345239 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:42:56.845224323 +0000 UTC m=+50.189878487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : configmap "kube-rbac-proxy" not found Jul 07 12:42:56.345376 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.345356 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96a07f6-2590-4338-8df3-4cfe79d22b4c-data-volume\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.354592 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.354563 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgqq\" (UniqueName: \"kubernetes.io/projected/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-api-access-slgqq\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.448771 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.448675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-56mk4" event={"ID":"b7392278-af8b-4f18-b906-0632aeda6a07","Type":"ContainerStarted","Data":"6c268800c296d335030fc127debe83b98b7427286a15b234ae85a38646733e68"} Jul 07 12:42:56.450410 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.450387 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:42:56.450696 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.450679 2578 scope.go:117] "RemoveContainer" containerID="6a8634670d304b369eaa16d470acd6782c520a02790edd1099d4cd0c25a437d3" Jul 07 12:42:56.450838 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.450819 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-d67d7_openshift-console-operator(0430f99a-b20d-4fb2-b994-1d150ef9cb30)\"" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" podUID="0430f99a-b20d-4fb2-b994-1d150ef9cb30" Jul 07 12:42:56.476690 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.476623 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-56mk4" podStartSLOduration=9.02185499 podStartE2EDuration="49.476601791s" totalImagesPullingTime="40.454746801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:42:56.475183192 +0000 UTC m=+49.819837371" watchObservedRunningTime="2026-07-07 12:42:56.476601791 +0000 UTC m=+49.821255976" Jul 07 12:42:56.849851 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.849813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.850074 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:56.849869 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:56.850074 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.849932 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 12:42:56.850074 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.849990 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 12:42:56.850074 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.849998 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:42:57.849981738 +0000 UTC m=+51.194635900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : configmap "kube-rbac-proxy" not found Jul 07 12:42:56.850074 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:56.850031 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:42:57.850019189 +0000 UTC m=+51.194673351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : secret "insights-runtime-extractor-tls" not found Jul 07 12:42:57.861454 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:57.861406 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:57.861856 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:57.861605 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 12:42:57.861856 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:57.861684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:57.861856 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:57.861700 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.86168026 +0000 UTC m=+53.206334436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : secret "insights-runtime-extractor-tls" not found Jul 07 12:42:57.861856 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:57.861736 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 12:42:57.861856 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:57.861781 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:42:59.861770792 +0000 UTC m=+53.206424953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : configmap "kube-rbac-proxy" not found Jul 07 12:42:58.752619 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:58.752589 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwwtx_71c2d709-99a0-43a4-82c0-632eed5c966a/dns-node-resolver/0.log" Jul 07 12:42:59.375836 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.375796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:59.375836 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.375845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.375874 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.375901 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.375939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.375958 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.375988 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376002 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd865c99-qfscz: secret "image-registry-tls" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376025 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376031 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:15.376010656 +0000 UTC m=+68.720664822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : secret "router-metrics-certs-default" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376070 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls podName:c25b375a-dac9-478a-b4ec-9a75fe8fd446 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:15.376059429 +0000 UTC m=+68.720713604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls") pod "image-registry-5cd865c99-qfscz" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446") : secret "image-registry-tls" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376026 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376083 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376091 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert podName:f1c22b51-fd94-4663-86d4-53c79d11996e nodeName:}" failed. No retries permitted until 2026-07-07 12:43:15.376080245 +0000 UTC m=+68.720734407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-67776" (UID: "f1c22b51-fd94-4663-86d4-53c79d11996e") : secret "networking-console-plugin-cert" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376143 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls podName:1acc0eb6-6234-43f8-ae0e-fda54a04ce53 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:15.376131724 +0000 UTC m=+68.720785891 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-6llv9" (UID: "1acc0eb6-6234-43f8-ae0e-fda54a04ce53") : secret "cluster-monitoring-operator-tls" not found Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.376173 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.376199 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:42:59.376322 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376241 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls podName:d2e4302f-b012-41ab-afc0-28c1f36f5389 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:15.376229825 +0000 UTC m=+68.720883992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls") pod "dns-default-4xbtp" (UID: "d2e4302f-b012-41ab-afc0-28c1f36f5389") : secret "dns-default-metrics-tls" not found Jul 07 12:42:59.376833 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376289 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 12:42:59.376833 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376320 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls podName:813dbf62-2e9f-458d-93aa-693b6c4bb673 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:15.376307478 +0000 UTC m=+68.720961639 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5hnzk" (UID: "813dbf62-2e9f-458d-93aa-693b6c4bb673") : secret "samples-operator-tls" not found Jul 07 12:42:59.376833 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.376378 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle podName:b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7 nodeName:}" failed. No retries permitted until 2026-07-07 12:43:15.376364624 +0000 UTC m=+68.721018800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle") pod "router-default-67ff897d47-8lqlg" (UID: "b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7") : configmap references non-existent config key: service-ca.crt Jul 07 12:42:59.760499 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.760428 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s27kt_3ac46ca0-b6cf-45d8-88ca-cd0553738b14/node-ca/0.log" Jul 07 12:42:59.880641 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.880601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:59.880824 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.880659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:42:59.880866 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.880836 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 12:42:59.880913 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:42:59.880902 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:43:03.880884075 +0000 UTC m=+57.225538239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : secret "insights-runtime-extractor-tls" not found Jul 07 12:42:59.881159 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:42:59.881142 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96a07f6-2590-4338-8df3-4cfe79d22b4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:43:00.151771 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:00.151738 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-26jzv_05a7a66a-0fe9-4c97-a899-d5c914212263/serve-healthcheck-canary/0.log" Jul 07 12:43:00.953696 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:00.953646 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-dkkf8_09307f15-ddb7-4d32-9af7-6021f032283e/kube-storage-version-migrator-operator/0.log" Jul 07 12:43:03.824871 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:03.824828 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:43:03.825286 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:03.824930 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:43:03.825286 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:03.825246 2578 scope.go:117] "RemoveContainer" containerID="6a8634670d304b369eaa16d470acd6782c520a02790edd1099d4cd0c25a437d3" Jul 07 12:43:03.825411 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:43:03.825390 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-d67d7_openshift-console-operator(0430f99a-b20d-4fb2-b994-1d150ef9cb30)\"" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" podUID="0430f99a-b20d-4fb2-b994-1d150ef9cb30" Jul 07 12:43:03.925703 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:03.925659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:43:03.925903 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:43:03.925876 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 12:43:03.925980 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:43:03.925964 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls podName:f96a07f6-2590-4338-8df3-4cfe79d22b4c nodeName:}" failed. No retries permitted until 2026-07-07 12:43:11.925943825 +0000 UTC m=+65.270597989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ht647" (UID: "f96a07f6-2590-4338-8df3-4cfe79d22b4c") : secret "insights-runtime-extractor-tls" not found Jul 07 12:43:04.476256 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:04.476228 2578 scope.go:117] "RemoveContainer" containerID="6a8634670d304b369eaa16d470acd6782c520a02790edd1099d4cd0c25a437d3" Jul 07 12:43:04.476426 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:43:04.476410 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-d67d7_openshift-console-operator(0430f99a-b20d-4fb2-b994-1d150ef9cb30)\"" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" podUID="0430f99a-b20d-4fb2-b994-1d150ef9cb30" Jul 07 12:43:08.361798 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:08.361767 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vs62b" Jul 07 12:43:11.993819 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:11.993779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:43:11.996140 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:11.996108 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96a07f6-2590-4338-8df3-4cfe79d22b4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ht647\" (UID: \"f96a07f6-2590-4338-8df3-4cfe79d22b4c\") " pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:43:12.099095 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.099063 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdzl4\"" Jul 07 12:43:12.107481 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.107459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ht647" Jul 07 12:43:12.239027 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.239006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ht647"] Jul 07 12:43:12.241513 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:43:12.241483 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96a07f6_2590_4338_8df3_4cfe79d22b4c.slice/crio-8b21b4b2ffbf6090318b149fedcb43ad9b6fc502ea97ed3bcb2d57cd0c3ca815 WatchSource:0}: Error finding container 8b21b4b2ffbf6090318b149fedcb43ad9b6fc502ea97ed3bcb2d57cd0c3ca815: Status 404 returned error can't find the container with id 8b21b4b2ffbf6090318b149fedcb43ad9b6fc502ea97ed3bcb2d57cd0c3ca815 Jul 07 12:43:12.506702 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.506615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ht647" event={"ID":"f96a07f6-2590-4338-8df3-4cfe79d22b4c","Type":"ContainerStarted","Data":"8b21b4b2ffbf6090318b149fedcb43ad9b6fc502ea97ed3bcb2d57cd0c3ca815"} Jul 07 12:43:12.802028 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.801927 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:43:12.804877 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.804850 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 12:43:12.816192 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.816158 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0cc952fb-050d-439d-a39a-3167617f0640-original-pull-secret\") pod \"global-pull-secret-syncer-22mxz\" (UID: \"0cc952fb-050d-439d-a39a-3167617f0640\") " pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:43:12.903523 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.902819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:43:12.903523 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.902977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:43:12.905968 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.905939 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 12:43:12.906884 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.906859 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql5md\" (UniqueName: \"kubernetes.io/projected/4ab45c9b-b63d-4436-a118-aca64a8af739-kube-api-access-ql5md\") pod \"network-check-target-lrs7r\" (UID: \"4ab45c9b-b63d-4436-a118-aca64a8af739\") " pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:43:12.916889 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.916853 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869f90b8-eca3-4c2f-9d03-db5f9207ae08-metrics-certs\") pod \"network-metrics-daemon-hjcq2\" (UID: \"869f90b8-eca3-4c2f-9d03-db5f9207ae08\") " pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:43:12.965813 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.965494 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-22mxz" Jul 07 12:43:12.975118 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.975091 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gxjtk\"" Jul 07 12:43:12.980931 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.980907 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-849mb\"" Jul 07 12:43:12.982992 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.982966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjcq2" Jul 07 12:43:12.988575 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:12.988428 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:43:13.162107 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.162067 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-22mxz"] Jul 07 12:43:13.168298 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:43:13.168246 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc952fb_050d_439d_a39a_3167617f0640.slice/crio-0adefc949cf23f5058e10b9d904bc9b4c4f8a2465c4dcffcf23038e9c0eb9bd8 WatchSource:0}: Error finding container 0adefc949cf23f5058e10b9d904bc9b4c4f8a2465c4dcffcf23038e9c0eb9bd8: Status 404 returned error can't find the container with id 0adefc949cf23f5058e10b9d904bc9b4c4f8a2465c4dcffcf23038e9c0eb9bd8 Jul 07 12:43:13.177074 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.177031 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjcq2"] Jul 07 12:43:13.182221 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:43:13.182150 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869f90b8_eca3_4c2f_9d03_db5f9207ae08.slice/crio-6e7f96278d3cf9b2fb134fcccb833cb9b5020583c268222d069f4fd8024bc508 WatchSource:0}: Error finding container 6e7f96278d3cf9b2fb134fcccb833cb9b5020583c268222d069f4fd8024bc508: Status 404 returned error can't find the container with id 6e7f96278d3cf9b2fb134fcccb833cb9b5020583c268222d069f4fd8024bc508 Jul 07 12:43:13.200691 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.200659 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lrs7r"] Jul 07 12:43:13.209460 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:43:13.209375 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab45c9b_b63d_4436_a118_aca64a8af739.slice/crio-73fbe9ea99e3967207f2047b99fbb1605d23a37bcc3a8cb72d469d2e6f5d1603 WatchSource:0}: Error finding container 73fbe9ea99e3967207f2047b99fbb1605d23a37bcc3a8cb72d469d2e6f5d1603: Status 404 returned error can't find the container with id 73fbe9ea99e3967207f2047b99fbb1605d23a37bcc3a8cb72d469d2e6f5d1603 Jul 07 12:43:13.511401 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.511302 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lrs7r" event={"ID":"4ab45c9b-b63d-4436-a118-aca64a8af739","Type":"ContainerStarted","Data":"317e939e1de1af1a99f976e186ebf3ca89a08fe06a89fa86b02f304a54c5055c"} Jul 07 12:43:13.511401 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.511366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lrs7r" event={"ID":"4ab45c9b-b63d-4436-a118-aca64a8af739","Type":"ContainerStarted","Data":"73fbe9ea99e3967207f2047b99fbb1605d23a37bcc3a8cb72d469d2e6f5d1603"} Jul 07 12:43:13.511638 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.511420 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:43:13.512572 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.512543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-22mxz" event={"ID":"0cc952fb-050d-439d-a39a-3167617f0640","Type":"ContainerStarted","Data":"0adefc949cf23f5058e10b9d904bc9b4c4f8a2465c4dcffcf23038e9c0eb9bd8"} Jul 07 12:43:13.513798 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.513770 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjcq2" event={"ID":"869f90b8-eca3-4c2f-9d03-db5f9207ae08","Type":"ContainerStarted","Data":"6e7f96278d3cf9b2fb134fcccb833cb9b5020583c268222d069f4fd8024bc508"} Jul 07 12:43:13.527481 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:13.527145 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lrs7r" podStartSLOduration=66.527125614 podStartE2EDuration="1m6.527125614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:13.526506414 +0000 UTC m=+66.871160600" watchObservedRunningTime="2026-07-07 12:43:13.527125614 +0000 UTC m=+66.871779793" Jul 07 12:43:15.428251 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.428200 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:43:15.428728 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.428261 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:15.428728 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.428297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:43:15.428728 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.428356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:15.428728 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.428396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:43:15.428728 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.428552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:43:15.428728 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.428618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:43:15.429106 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.429079 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-service-ca-bundle\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:15.431462 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.431435 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"image-registry-5cd865c99-qfscz\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:43:15.431750 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.431647 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7-metrics-certs\") pod \"router-default-67ff897d47-8lqlg\" (UID: \"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7\") " pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:15.432157 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.432108 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f1c22b51-fd94-4663-86d4-53c79d11996e-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-67776\" (UID: \"f1c22b51-fd94-4663-86d4-53c79d11996e\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:43:15.432288 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.432263 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2e4302f-b012-41ab-afc0-28c1f36f5389-metrics-tls\") pod \"dns-default-4xbtp\" (UID: \"d2e4302f-b012-41ab-afc0-28c1f36f5389\") " pod="openshift-dns/dns-default-4xbtp" Jul 07 12:43:15.432364 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.432326 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/1acc0eb6-6234-43f8-ae0e-fda54a04ce53-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-6llv9\" (UID: \"1acc0eb6-6234-43f8-ae0e-fda54a04ce53\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:43:15.432445 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.432418 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/813dbf62-2e9f-458d-93aa-693b6c4bb673-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5hnzk\" (UID: \"813dbf62-2e9f-458d-93aa-693b6c4bb673\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:43:15.580094 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.580064 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8ss65\"" Jul 07 12:43:15.587455 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.587396 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:43:15.618976 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.618947 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ncl7q\"" Jul 07 12:43:15.627289 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.627257 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4xbtp" Jul 07 12:43:15.640322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.640294 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nfxgv\"" Jul 07 12:43:15.647810 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.647787 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" Jul 07 12:43:15.648358 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.648340 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-z24dh\"" Jul 07 12:43:15.654756 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.654705 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xdn77\"" Jul 07 12:43:15.656785 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.656765 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" Jul 07 12:43:15.661330 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.661279 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6x89k\"" Jul 07 12:43:15.662354 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.662326 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:15.668452 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:15.668429 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" Jul 07 12:43:18.156312 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.156074 2578 scope.go:117] "RemoveContainer" containerID="6a8634670d304b369eaa16d470acd6782c520a02790edd1099d4cd0c25a437d3" Jul 07 12:43:18.264194 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.263788 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cd865c99-qfscz"] Jul 07 12:43:18.280976 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.278392 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-67776"] Jul 07 12:43:18.280976 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.278451 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4xbtp"] Jul 07 12:43:18.284443 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.284419 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk"] Jul 07 12:43:18.315162 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.314863 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67ff897d47-8lqlg"] Jul 07 12:43:18.322973 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.322896 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9"] Jul 07 12:43:18.536743 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.536655 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" event={"ID":"1acc0eb6-6234-43f8-ae0e-fda54a04ce53","Type":"ContainerStarted","Data":"0df10bb7ac266a46915324b2ac0f7cde14f96b60db188972f5cdcad77eaf62c5"} Jul 07 12:43:18.538552 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.538520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67ff897d47-8lqlg" event={"ID":"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7","Type":"ContainerStarted","Data":"ff3d3098e36a4039410fee4a5dc157bacc4459ee18a2beb3c4de2af06037e356"} Jul 07 12:43:18.538763 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.538744 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67ff897d47-8lqlg" event={"ID":"b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7","Type":"ContainerStarted","Data":"659661fdadaf449da63dc1dc214ba5034422b8c28dc9ffda49723b1f487641ce"} Jul 07 12:43:18.539936 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.539906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" event={"ID":"813dbf62-2e9f-458d-93aa-693b6c4bb673","Type":"ContainerStarted","Data":"75436e66e4df8d8ca5c07c0ffe44fa94346772d5d4bc9145a957e00008c67588"} Jul 07 12:43:18.541291 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.541264 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" event={"ID":"f1c22b51-fd94-4663-86d4-53c79d11996e","Type":"ContainerStarted","Data":"e230cdd89e608fbd732230f63d6a64e5cbf7b0066ac64b751db7c7f4f1c9d4a7"} Jul 07 12:43:18.543472 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.543445 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" event={"ID":"c25b375a-dac9-478a-b4ec-9a75fe8fd446","Type":"ContainerStarted","Data":"1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f"} Jul 07 12:43:18.543578 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.543495 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" event={"ID":"c25b375a-dac9-478a-b4ec-9a75fe8fd446","Type":"ContainerStarted","Data":"f24a59483083b7a300364c2f6c425d28e318843d90870a8cc2a6b78592547ef0"} Jul 07 12:43:18.543643 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.543585 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:43:18.545178 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.545156 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-22mxz" event={"ID":"0cc952fb-050d-439d-a39a-3167617f0640","Type":"ContainerStarted","Data":"58d07560c041c156766876d910f2b0de59ec100c3180229f588a3d12d6cf54d9"} Jul 07 12:43:18.547421 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.547404 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:43:18.547527 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.547475 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" event={"ID":"0430f99a-b20d-4fb2-b994-1d150ef9cb30","Type":"ContainerStarted","Data":"e6459ee1a40f243aad970fd1492c7f999a4f041f996cf2e319eb788d2eec14d0"} Jul 07 12:43:18.547782 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.547767 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:43:18.549349 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.549327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ht647" event={"ID":"f96a07f6-2590-4338-8df3-4cfe79d22b4c","Type":"ContainerStarted","Data":"b943071992862e4dddb5aafce8f39a697ff0a993074b64ac579dd7c661695cf6"} Jul 07 12:43:18.551742 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.551715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjcq2" event={"ID":"869f90b8-eca3-4c2f-9d03-db5f9207ae08","Type":"ContainerStarted","Data":"69ac2272a3f2f8bd2a52d55117b29afed6627a0b559fbab4953a4a9d659d5c0f"} Jul 07 12:43:18.551867 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.551764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjcq2" event={"ID":"869f90b8-eca3-4c2f-9d03-db5f9207ae08","Type":"ContainerStarted","Data":"0812b64274250f9483ca8499cb79e56b60e87d412e7b67ca1245e2dc0f17ed85"} Jul 07 12:43:18.553182 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.553126 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xbtp" event={"ID":"d2e4302f-b012-41ab-afc0-28c1f36f5389","Type":"ContainerStarted","Data":"534e88d0b71ecf4c22f792d641fabe93421001fdbded0ccfdfa60ff9eb4b0213"} Jul 07 12:43:18.560391 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.560329 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-67ff897d47-8lqlg" podStartSLOduration=41.560306312 podStartE2EDuration="41.560306312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:18.559619874 +0000 UTC m=+71.904274060" watchObservedRunningTime="2026-07-07 12:43:18.560306312 +0000 UTC m=+71.904960494" Jul 07 12:43:18.575014 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.574947 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-22mxz" podStartSLOduration=66.754386095 podStartE2EDuration="1m11.574925572s" totalImagesPullingTime="4.820539477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:18.574241199 +0000 UTC m=+71.918895385" watchObservedRunningTime="2026-07-07 12:43:18.574925572 +0000 UTC m=+71.919579757" Jul 07 12:43:18.594750 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.594685 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" podStartSLOduration=71.594665803 podStartE2EDuration="1m11.594665803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:18.594547283 +0000 UTC m=+71.939201470" watchObservedRunningTime="2026-07-07 12:43:18.594665803 +0000 UTC m=+71.939319989" Jul 07 12:43:18.614667 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.614606 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hjcq2" podStartSLOduration=66.837790375 podStartE2EDuration="1m11.614588258s" totalImagesPullingTime="4.776797883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:18.613943758 +0000 UTC m=+71.958597943" watchObservedRunningTime="2026-07-07 12:43:18.614588258 +0000 UTC m=+71.959242443" Jul 07 12:43:18.631177 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.631117 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" podStartSLOduration=31.88113711 podStartE2EDuration="41.631095486s" totalImagesPullingTime="9.749958376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:18.630527829 +0000 UTC m=+71.975182014" watchObservedRunningTime="2026-07-07 12:43:18.631095486 +0000 UTC m=+71.975749674" Jul 07 12:43:18.662731 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.662700 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:18.665571 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.665546 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:18.788911 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:18.788200 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-d67d7" Jul 07 12:43:19.559585 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:19.559501 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:19.561717 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:19.561487 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-67ff897d47-8lqlg" Jul 07 12:43:20.969232 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:20.969190 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5"] Jul 07 12:43:21.008244 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.008206 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5"] Jul 07 12:43:21.008408 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.008324 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.011113 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.011082 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 07 12:43:21.012570 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.012545 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 12:43:21.012570 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.012567 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 12:43:21.012719 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.012590 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 12:43:21.012719 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.012609 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 07 12:43:21.012719 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.012553 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 07 12:43:21.012719 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.012549 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 07 12:43:21.186476 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.186380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhgvp\" (UniqueName: \"kubernetes.io/projected/645f5830-4a09-40c9-a988-042f4cb127b0-kube-api-access-qhgvp\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.186476 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.186451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.186476 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.186475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-hub\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.186742 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.186537 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-ca\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.186742 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.186562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/645f5830-4a09-40c9-a988-042f4cb127b0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.186742 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.186590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.287929 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.287897 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhgvp\" (UniqueName: \"kubernetes.io/projected/645f5830-4a09-40c9-a988-042f4cb127b0-kube-api-access-qhgvp\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.288104 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.287939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.288104 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.287960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-hub\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.288104 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.288016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-ca\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.288104 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.288058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/645f5830-4a09-40c9-a988-042f4cb127b0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.288313 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.288194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.288849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.288821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/645f5830-4a09-40c9-a988-042f4cb127b0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.290657 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.290629 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-ca\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.290806 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.290711 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.290901 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.290881 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.291244 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.291229 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/645f5830-4a09-40c9-a988-042f4cb127b0-hub\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.303638 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.303610 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhgvp\" (UniqueName: \"kubernetes.io/projected/645f5830-4a09-40c9-a988-042f4cb127b0-kube-api-access-qhgvp\") pod \"cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5\" (UID: \"645f5830-4a09-40c9-a988-042f4cb127b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:21.337982 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:21.337947 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" Jul 07 12:43:22.343837 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.343512 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5"] Jul 07 12:43:22.358607 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:43:22.358566 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod645f5830_4a09_40c9_a988_042f4cb127b0.slice/crio-d644e5ee2d60f6266650c7c3ce02bde76b8d7a2860b0d0d81386abd0cbfde524 WatchSource:0}: Error finding container d644e5ee2d60f6266650c7c3ce02bde76b8d7a2860b0d0d81386abd0cbfde524: Status 404 returned error can't find the container with id d644e5ee2d60f6266650c7c3ce02bde76b8d7a2860b0d0d81386abd0cbfde524 Jul 07 12:43:22.574720 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.574679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ht647" event={"ID":"f96a07f6-2590-4338-8df3-4cfe79d22b4c","Type":"ContainerStarted","Data":"96bac1edb7d747f8ebed2711a64f334b1ef5d8f34782dc1f09a4df7e6b63ac0e"} Jul 07 12:43:22.575894 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.575861 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" event={"ID":"645f5830-4a09-40c9-a988-042f4cb127b0","Type":"ContainerStarted","Data":"d644e5ee2d60f6266650c7c3ce02bde76b8d7a2860b0d0d81386abd0cbfde524"} Jul 07 12:43:22.577514 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.577482 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xbtp" event={"ID":"d2e4302f-b012-41ab-afc0-28c1f36f5389","Type":"ContainerStarted","Data":"cab03a013f62594978533f2471ecc54d6501380e272b9d620c5a0eb2cd24efd9"} Jul 07 12:43:22.579356 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.579328 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" event={"ID":"813dbf62-2e9f-458d-93aa-693b6c4bb673","Type":"ContainerStarted","Data":"38d3d5c270de1e88fcfe7f019cdebc92016172c30a5bdb3e20c7d5ba143a9029"} Jul 07 12:43:22.579504 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.579378 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" event={"ID":"813dbf62-2e9f-458d-93aa-693b6c4bb673","Type":"ContainerStarted","Data":"0c727385e20600b305463b725bd43d0634ffdecd698e2b9976c7bbc3eff4f7b1"} Jul 07 12:43:22.581108 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.581083 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" event={"ID":"f1c22b51-fd94-4663-86d4-53c79d11996e","Type":"ContainerStarted","Data":"b8fd30ad0c8774b0256a58113ffd07547b88af2ce294b6de573e1512cc3e4edf"} Jul 07 12:43:22.582845 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.582819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" event={"ID":"1acc0eb6-6234-43f8-ae0e-fda54a04ce53","Type":"ContainerStarted","Data":"390aabb03da59a2ae1fac781173f1a147f8521548e32d239c7f49c050b049ba3"} Jul 07 12:43:22.604252 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.604199 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5hnzk" podStartSLOduration=41.75376964 podStartE2EDuration="45.604178812s" totalImagesPullingTime="3.850409172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:22.603630781 +0000 UTC m=+75.948284968" watchObservedRunningTime="2026-07-07 12:43:22.604178812 +0000 UTC m=+75.948832977" Jul 07 12:43:22.622242 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.622181 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-67776" podStartSLOduration=38.72175354 podStartE2EDuration="42.622160273s" totalImagesPullingTime="3.900406733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:22.621490699 +0000 UTC m=+75.966144884" watchObservedRunningTime="2026-07-07 12:43:22.622160273 +0000 UTC m=+75.966814458" Jul 07 12:43:22.640828 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:22.640302 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-6llv9" podStartSLOduration=41.776575691 podStartE2EDuration="45.640275645s" totalImagesPullingTime="3.863699954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:22.638560407 +0000 UTC m=+75.983214594" watchObservedRunningTime="2026-07-07 12:43:22.640275645 +0000 UTC m=+75.984929829" Jul 07 12:43:23.591266 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:23.591188 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4xbtp" event={"ID":"d2e4302f-b012-41ab-afc0-28c1f36f5389","Type":"ContainerStarted","Data":"484375325e8befa67665a115b6a3d7c04fa9d9994ec0d100033306c18417364f"} Jul 07 12:43:23.610429 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:23.610305 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4xbtp" podStartSLOduration=36.705417975 podStartE2EDuration="40.61028319s" totalImagesPullingTime="3.904865215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:23.609121589 +0000 UTC m=+76.953775773" watchObservedRunningTime="2026-07-07 12:43:23.61028319 +0000 UTC m=+76.954937374" Jul 07 12:43:24.596125 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:24.596091 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4xbtp" Jul 07 12:43:28.614430 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:28.614382 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ht647" event={"ID":"f96a07f6-2590-4338-8df3-4cfe79d22b4c","Type":"ContainerStarted","Data":"2482cbc0b071a001706172f1e084734495f722491054d930080d0d9abcd39b0e"} Jul 07 12:43:28.616661 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:28.616630 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" event={"ID":"645f5830-4a09-40c9-a988-042f4cb127b0","Type":"ContainerStarted","Data":"dadadf9170ccf69c6648ccf417958feec2f666260e3f9af9eea9df3ffb8c9727"} Jul 07 12:43:28.616786 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:28.616670 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" event={"ID":"645f5830-4a09-40c9-a988-042f4cb127b0","Type":"ContainerStarted","Data":"a20d2ad92da0132b71b6bc2690d8f645422261759d994a133bb2aac1705a5984"} Jul 07 12:43:28.616786 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:28.616681 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" event={"ID":"645f5830-4a09-40c9-a988-042f4cb127b0","Type":"ContainerStarted","Data":"1cb0db90bf25aa3a2795c8f44c8610e4cd7c4a0fc86e87b6710d01cac0dc4c39"} Jul 07 12:43:28.639076 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:28.639004 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ht647" podStartSLOduration=17.819514213 podStartE2EDuration="32.638988609s" totalImagesPullingTime="14.819474396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:42:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:28.638287847 +0000 UTC m=+81.982942031" watchObservedRunningTime="2026-07-07 12:43:28.638988609 +0000 UTC m=+81.983642799" Jul 07 12:43:28.657677 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:28.657627 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bcf9cc9b-zz8x5" podStartSLOduration=3.165066354 podStartE2EDuration="8.657611551s" totalImagesPullingTime="5.492545197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:28.656423365 +0000 UTC m=+82.001077550" watchObservedRunningTime="2026-07-07 12:43:28.657611551 +0000 UTC m=+82.002265736" Jul 07 12:43:34.602506 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:34.602389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4xbtp" Jul 07 12:43:35.592558 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:35.592521 2578 patch_prober.go:28] interesting pod/image-registry-5cd865c99-qfscz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 12:43:35.592730 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:35.592577 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" podUID="c25b375a-dac9-478a-b4ec-9a75fe8fd446" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 12:43:38.249185 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.249146 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-95sts"] Jul 07 12:43:38.252124 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.252103 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.255015 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.254994 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 12:43:38.255173 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.255113 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 12:43:38.255333 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.255320 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-w4kcv\"" Jul 07 12:43:38.255382 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.255327 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 12:43:38.255382 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.255361 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 12:43:38.330877 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.330843 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-textfile\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331080 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.330887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-wtmp\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331080 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.330910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbwjr\" (UniqueName: \"kubernetes.io/projected/14d0298b-2c83-4e97-8d37-36588e79000f-kube-api-access-rbwjr\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331080 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.330975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-root\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331080 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.331030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-tls\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331243 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.331112 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-sys\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331243 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.331159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-accelerators-collector-config\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331243 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.331184 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14d0298b-2c83-4e97-8d37-36588e79000f-metrics-client-ca\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.331243 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.331201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.432542 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-accelerators-collector-config\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.432542 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14d0298b-2c83-4e97-8d37-36588e79000f-metrics-client-ca\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.432849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432566 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.432849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-textfile\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.432849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-wtmp\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.432849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432820 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbwjr\" (UniqueName: \"kubernetes.io/projected/14d0298b-2c83-4e97-8d37-36588e79000f-kube-api-access-rbwjr\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.432849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-root\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433130 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-tls\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433130 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432972 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-wtmp\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433130 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432989 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-root\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433130 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.432975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-sys\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433130 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.433007 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14d0298b-2c83-4e97-8d37-36588e79000f-sys\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433338 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:43:38.433151 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 12:43:38.433338 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:43:38.433217 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-tls podName:14d0298b-2c83-4e97-8d37-36588e79000f nodeName:}" failed. No retries permitted until 2026-07-07 12:43:38.933199591 +0000 UTC m=+92.277853773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-tls") pod "node-exporter-95sts" (UID: "14d0298b-2c83-4e97-8d37-36588e79000f") : secret "node-exporter-tls" not found Jul 07 12:43:38.433338 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.433151 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-textfile\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433338 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.433281 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-accelerators-collector-config\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.433473 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.433420 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14d0298b-2c83-4e97-8d37-36588e79000f-metrics-client-ca\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.435116 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.435095 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.444095 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.444031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbwjr\" (UniqueName: \"kubernetes.io/projected/14d0298b-2c83-4e97-8d37-36588e79000f-kube-api-access-rbwjr\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.937620 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.937584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-tls\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:38.940165 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:38.940138 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14d0298b-2c83-4e97-8d37-36588e79000f-node-exporter-tls\") pod \"node-exporter-95sts\" (UID: \"14d0298b-2c83-4e97-8d37-36588e79000f\") " pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:39.161324 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:39.161294 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95sts" Jul 07 12:43:39.173713 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:43:39.173678 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14d0298b_2c83_4e97_8d37_36588e79000f.slice/crio-ac716c888ffa74c019b22027b5eccbd4086575329638a1246b4b52a62f351e1f WatchSource:0}: Error finding container ac716c888ffa74c019b22027b5eccbd4086575329638a1246b4b52a62f351e1f: Status 404 returned error can't find the container with id ac716c888ffa74c019b22027b5eccbd4086575329638a1246b4b52a62f351e1f Jul 07 12:43:39.564785 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:39.564758 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:43:39.660173 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:39.660135 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95sts" event={"ID":"14d0298b-2c83-4e97-8d37-36588e79000f","Type":"ContainerStarted","Data":"ac716c888ffa74c019b22027b5eccbd4086575329638a1246b4b52a62f351e1f"} Jul 07 12:43:40.665136 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:40.665095 2578 generic.go:356] "Generic (PLEG): container finished" podID="14d0298b-2c83-4e97-8d37-36588e79000f" containerID="915be02d8ee2f5e59f798996c6948971a68e338eeb3e5ec27706d6f739b3f23a" exitCode=0 Jul 07 12:43:40.665605 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:40.665190 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95sts" event={"ID":"14d0298b-2c83-4e97-8d37-36588e79000f","Type":"ContainerDied","Data":"915be02d8ee2f5e59f798996c6948971a68e338eeb3e5ec27706d6f739b3f23a"} Jul 07 12:43:41.670826 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:41.670791 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95sts" event={"ID":"14d0298b-2c83-4e97-8d37-36588e79000f","Type":"ContainerStarted","Data":"bb9cee88d3e486865b192954302c203374cda7c12aee089b0d01e5d01cb0c998"} Jul 07 12:43:41.671247 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:41.670847 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95sts" event={"ID":"14d0298b-2c83-4e97-8d37-36588e79000f","Type":"ContainerStarted","Data":"f19a0a67b8c5cb8c459c7959be7ecdac4c2d7f587fd8d5e6cf84814106ad93a4"} Jul 07 12:43:41.696679 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:41.696626 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-95sts" podStartSLOduration=2.693672247 podStartE2EDuration="3.696608843s" totalImagesPullingTime="935.743722ms" totalInitContainerRuntime="67.192874ms" isStatefulPod=true podCreationTimestamp="2026-07-07 12:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:43:41.694409688 +0000 UTC m=+95.039063873" watchObservedRunningTime="2026-07-07 12:43:41.696608843 +0000 UTC m=+95.041263027" Jul 07 12:43:44.520007 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:44.519978 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lrs7r" Jul 07 12:43:45.058030 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:43:45.057993 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cd865c99-qfscz"] Jul 07 12:44:10.078473 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.078414 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" podUID="c25b375a-dac9-478a-b4ec-9a75fe8fd446" containerName="registry" containerID="cri-o://1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f" gracePeriod=30 Jul 07 12:44:10.322840 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.322815 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:44:10.416415 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416314 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-image-registry-private-configuration\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416415 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416366 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-certificates\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416415 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416393 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-trusted-ca\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416415 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416421 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sz88\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-kube-api-access-5sz88\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416744 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416438 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c25b375a-dac9-478a-b4ec-9a75fe8fd446-ca-trust-extracted\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416744 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416454 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-installation-pull-secrets\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416744 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416481 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-bound-sa-token\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416744 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416517 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") pod \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\" (UID: \"c25b375a-dac9-478a-b4ec-9a75fe8fd446\") " Jul 07 12:44:10.416976 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416932 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:10.416976 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.416952 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 12:44:10.419131 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.419097 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:10.419281 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.419135 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:44:10.419281 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.419093 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:44:10.419281 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.419233 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-kube-api-access-5sz88" (OuterVolumeSpecName: "kube-api-access-5sz88") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "kube-api-access-5sz88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 12:44:10.419453 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.419327 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 12:44:10.426181 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.426144 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c25b375a-dac9-478a-b4ec-9a75fe8fd446-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c25b375a-dac9-478a-b4ec-9a75fe8fd446" (UID: "c25b375a-dac9-478a-b4ec-9a75fe8fd446"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 12:44:10.517898 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517855 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sz88\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-kube-api-access-5sz88\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.517898 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517888 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c25b375a-dac9-478a-b4ec-9a75fe8fd446-ca-trust-extracted\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.517898 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517898 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-installation-pull-secrets\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.517898 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517907 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-bound-sa-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.518198 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517916 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-tls\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.518198 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517926 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c25b375a-dac9-478a-b4ec-9a75fe8fd446-image-registry-private-configuration\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.518198 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517934 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-registry-certificates\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.518198 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.517943 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c25b375a-dac9-478a-b4ec-9a75fe8fd446-trusted-ca\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 12:44:10.780493 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.780413 2578 generic.go:356] "Generic (PLEG): container finished" podID="c25b375a-dac9-478a-b4ec-9a75fe8fd446" containerID="1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f" exitCode=0 Jul 07 12:44:10.780493 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.780460 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" Jul 07 12:44:10.780493 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.780474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" event={"ID":"c25b375a-dac9-478a-b4ec-9a75fe8fd446","Type":"ContainerDied","Data":"1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f"} Jul 07 12:44:10.780722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.780513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd865c99-qfscz" event={"ID":"c25b375a-dac9-478a-b4ec-9a75fe8fd446","Type":"ContainerDied","Data":"f24a59483083b7a300364c2f6c425d28e318843d90870a8cc2a6b78592547ef0"} Jul 07 12:44:10.780722 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.780534 2578 scope.go:117] "RemoveContainer" containerID="1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f" Jul 07 12:44:10.789021 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.789005 2578 scope.go:117] "RemoveContainer" containerID="1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f" Jul 07 12:44:10.789307 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:44:10.789279 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f\": container with ID starting with 1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f not found: ID does not exist" containerID="1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f" Jul 07 12:44:10.789403 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.789312 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f"} err="failed to get container status \"1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f\": rpc error: code = NotFound desc = could not find container \"1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f\": container with ID starting with 1411912ebe7154bdaa42b84b2470f5a1c9b2be22fc090f85542c0d3800abbe2f not found: ID does not exist" Jul 07 12:44:10.802750 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.802725 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cd865c99-qfscz"] Jul 07 12:44:10.806033 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:10.806011 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cd865c99-qfscz"] Jul 07 12:44:11.155701 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:11.155664 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c25b375a-dac9-478a-b4ec-9a75fe8fd446" path="/var/lib/kubelet/pods/c25b375a-dac9-478a-b4ec-9a75fe8fd446/volumes" Jul 07 12:44:15.800169 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:15.800133 2578 generic.go:356] "Generic (PLEG): container finished" podID="08c14ad2-d6d6-4f75-a63b-6db8b8714b39" containerID="285676327c97667a42f57c213cbbacf312de153f88747ed9853803cf80b62676" exitCode=0 Jul 07 12:44:15.800637 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:15.800209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" event={"ID":"08c14ad2-d6d6-4f75-a63b-6db8b8714b39","Type":"ContainerDied","Data":"285676327c97667a42f57c213cbbacf312de153f88747ed9853803cf80b62676"} Jul 07 12:44:15.800637 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:15.800555 2578 scope.go:117] "RemoveContainer" containerID="285676327c97667a42f57c213cbbacf312de153f88747ed9853803cf80b62676" Jul 07 12:44:16.805297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:16.805259 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-c6fll" event={"ID":"08c14ad2-d6d6-4f75-a63b-6db8b8714b39","Type":"ContainerStarted","Data":"5b2ebc56db1574da0f6b6d3a646499fc949c60dca200bacb9115eda4d4953a49"} Jul 07 12:44:20.820963 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:20.820931 2578 generic.go:356] "Generic (PLEG): container finished" podID="53e4e2b2-0715-449f-b683-43191049e45a" containerID="036137661a4dc7cb38d3e9b1c57b96633c174505f50a16d3741f8b4428acc661" exitCode=0 Jul 07 12:44:20.821465 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:20.821005 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" event={"ID":"53e4e2b2-0715-449f-b683-43191049e45a","Type":"ContainerDied","Data":"036137661a4dc7cb38d3e9b1c57b96633c174505f50a16d3741f8b4428acc661"} Jul 07 12:44:20.821465 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:20.821416 2578 scope.go:117] "RemoveContainer" containerID="036137661a4dc7cb38d3e9b1c57b96633c174505f50a16d3741f8b4428acc661" Jul 07 12:44:20.822515 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:20.822487 2578 generic.go:356] "Generic (PLEG): container finished" podID="09307f15-ddb7-4d32-9af7-6021f032283e" containerID="d5f5a002dfa617c9c8a10c51ecf87b31577afbcfbc8ff64cf4df41cccf5d1324" exitCode=0 Jul 07 12:44:20.822615 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:20.822545 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" event={"ID":"09307f15-ddb7-4d32-9af7-6021f032283e","Type":"ContainerDied","Data":"d5f5a002dfa617c9c8a10c51ecf87b31577afbcfbc8ff64cf4df41cccf5d1324"} Jul 07 12:44:20.822806 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:20.822794 2578 scope.go:117] "RemoveContainer" containerID="d5f5a002dfa617c9c8a10c51ecf87b31577afbcfbc8ff64cf4df41cccf5d1324" Jul 07 12:44:21.828547 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:21.828509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ckz8l" event={"ID":"53e4e2b2-0715-449f-b683-43191049e45a","Type":"ContainerStarted","Data":"fdb11e2376ca367356491a35e12922cb5372bcc12823e88f61aca10fc0d23cf5"} Jul 07 12:44:21.830256 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:44:21.830226 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-dkkf8" event={"ID":"09307f15-ddb7-4d32-9af7-6021f032283e","Type":"ContainerStarted","Data":"d637e70b8c7b501f01c76ace9c7ac3dcec262899430e6e851e482299924aaa54"} Jul 07 12:47:07.017594 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:07.017553 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:47:07.023160 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:07.023134 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:47:07.028516 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:07.028488 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 12:47:39.982718 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.982678 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r"] Jul 07 12:47:39.983990 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.982971 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25b375a-dac9-478a-b4ec-9a75fe8fd446" containerName="registry" Jul 07 12:47:39.983990 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.982981 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25b375a-dac9-478a-b4ec-9a75fe8fd446" containerName="registry" Jul 07 12:47:39.983990 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.983058 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c25b375a-dac9-478a-b4ec-9a75fe8fd446" containerName="registry" Jul 07 12:47:39.984705 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.984688 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:39.987946 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.987919 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-hlrbq\"" Jul 07 12:47:39.988203 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.988187 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:47:39.988308 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:39.988250 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 12:47:40.003999 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.003964 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r"] Jul 07 12:47:40.022857 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.022817 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbqt\" (UniqueName: \"kubernetes.io/projected/d7439aef-d6a2-416f-acb5-1fda269b01ec-kube-api-access-7kbqt\") pod \"cert-manager-operator-controller-manager-55c444bf86-2zd2r\" (UID: \"d7439aef-d6a2-416f-acb5-1fda269b01ec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:40.023036 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.022871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7439aef-d6a2-416f-acb5-1fda269b01ec-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2zd2r\" (UID: \"d7439aef-d6a2-416f-acb5-1fda269b01ec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:40.123800 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.123762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbqt\" (UniqueName: \"kubernetes.io/projected/d7439aef-d6a2-416f-acb5-1fda269b01ec-kube-api-access-7kbqt\") pod \"cert-manager-operator-controller-manager-55c444bf86-2zd2r\" (UID: \"d7439aef-d6a2-416f-acb5-1fda269b01ec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:40.123983 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.123812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7439aef-d6a2-416f-acb5-1fda269b01ec-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2zd2r\" (UID: \"d7439aef-d6a2-416f-acb5-1fda269b01ec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:40.124236 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.124220 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7439aef-d6a2-416f-acb5-1fda269b01ec-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2zd2r\" (UID: \"d7439aef-d6a2-416f-acb5-1fda269b01ec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:40.135075 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.135028 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbqt\" (UniqueName: \"kubernetes.io/projected/d7439aef-d6a2-416f-acb5-1fda269b01ec-kube-api-access-7kbqt\") pod \"cert-manager-operator-controller-manager-55c444bf86-2zd2r\" (UID: \"d7439aef-d6a2-416f-acb5-1fda269b01ec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:40.293812 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.293727 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" Jul 07 12:47:40.433177 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.433153 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r"] Jul 07 12:47:40.436301 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:47:40.436273 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7439aef_d6a2_416f_acb5_1fda269b01ec.slice/crio-12c0ebef8f085e769b72345009d74160ec07a24bbd17e33c42f90dd2e1d8d150 WatchSource:0}: Error finding container 12c0ebef8f085e769b72345009d74160ec07a24bbd17e33c42f90dd2e1d8d150: Status 404 returned error can't find the container with id 12c0ebef8f085e769b72345009d74160ec07a24bbd17e33c42f90dd2e1d8d150 Jul 07 12:47:40.439214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.439196 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:47:40.563523 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:40.563435 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" event={"ID":"d7439aef-d6a2-416f-acb5-1fda269b01ec","Type":"ContainerStarted","Data":"12c0ebef8f085e769b72345009d74160ec07a24bbd17e33c42f90dd2e1d8d150"} Jul 07 12:47:43.579532 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:47:43.579489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" event={"ID":"d7439aef-d6a2-416f-acb5-1fda269b01ec","Type":"ContainerStarted","Data":"b8f1fb3a3e1e68c8cc049910c32a86410c29548ff9500d2d0716b3c855922781"} Jul 07 12:48:16.962795 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.962687 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2zd2r" podStartSLOduration=35.288319886 podStartE2EDuration="37.962667166s" totalImagesPullingTime="2.67434728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:47:43.620442029 +0000 UTC m=+336.965096213" watchObservedRunningTime="2026-07-07 12:48:16.962667166 +0000 UTC m=+370.307321350" Jul 07 12:48:16.963326 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.963028 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b"] Jul 07 12:48:16.987080 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.987024 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:16.990982 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.990954 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 12:48:16.990982 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.990969 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 12:48:16.991686 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.991672 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 12:48:16.992761 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.992743 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 12:48:16.996904 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:16.996885 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7tjqs\"" Jul 07 12:48:17.029812 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.029774 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b"] Jul 07 12:48:17.134832 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.134791 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a44e2e6b-06a6-4cab-b490-783fdab05203-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.134832 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.134834 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a44e2e6b-06a6-4cab-b490-783fdab05203-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.135130 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.134919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a44e2e6b-06a6-4cab-b490-783fdab05203-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.135130 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.135072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j5wx\" (UniqueName: \"kubernetes.io/projected/a44e2e6b-06a6-4cab-b490-783fdab05203-kube-api-access-5j5wx\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.135223 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.135128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a44e2e6b-06a6-4cab-b490-783fdab05203-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.235868 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.235776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a44e2e6b-06a6-4cab-b490-783fdab05203-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.235868 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.235836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j5wx\" (UniqueName: \"kubernetes.io/projected/a44e2e6b-06a6-4cab-b490-783fdab05203-kube-api-access-5j5wx\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.235868 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.235862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a44e2e6b-06a6-4cab-b490-783fdab05203-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.236187 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.235880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a44e2e6b-06a6-4cab-b490-783fdab05203-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.236187 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.235898 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a44e2e6b-06a6-4cab-b490-783fdab05203-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.236410 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.236388 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a44e2e6b-06a6-4cab-b490-783fdab05203-manifests\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.238323 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.238290 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a44e2e6b-06a6-4cab-b490-783fdab05203-tmp\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.238529 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.238510 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a44e2e6b-06a6-4cab-b490-783fdab05203-webhook-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.238630 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.238610 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a44e2e6b-06a6-4cab-b490-783fdab05203-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.277348 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.277313 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j5wx\" (UniqueName: \"kubernetes.io/projected/a44e2e6b-06a6-4cab-b490-783fdab05203-kube-api-access-5j5wx\") pod \"opendatahub-operator-controller-manager-68cfd86456-rgm4b\" (UID: \"a44e2e6b-06a6-4cab-b490-783fdab05203\") " pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.297232 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.297195 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:17.474551 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.474506 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b"] Jul 07 12:48:17.477900 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:48:17.477863 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44e2e6b_06a6_4cab_b490_783fdab05203.slice/crio-d457bde84616fc29395dfe210924a13ff5d5642f1bfa251dad5b16c31720f4f5 WatchSource:0}: Error finding container d457bde84616fc29395dfe210924a13ff5d5642f1bfa251dad5b16c31720f4f5: Status 404 returned error can't find the container with id d457bde84616fc29395dfe210924a13ff5d5642f1bfa251dad5b16c31720f4f5 Jul 07 12:48:17.722910 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:17.722872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" event={"ID":"a44e2e6b-06a6-4cab-b490-783fdab05203","Type":"ContainerStarted","Data":"d457bde84616fc29395dfe210924a13ff5d5642f1bfa251dad5b16c31720f4f5"} Jul 07 12:48:21.744285 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:21.744244 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" event={"ID":"a44e2e6b-06a6-4cab-b490-783fdab05203","Type":"ContainerStarted","Data":"c2ca979638c802843fbc20aeba93b9ac655c91e137945aa66fb6f303c9536c6c"} Jul 07 12:48:22.750443 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:22.750407 2578 generic.go:356] "Generic (PLEG): container finished" podID="a44e2e6b-06a6-4cab-b490-783fdab05203" containerID="c2ca979638c802843fbc20aeba93b9ac655c91e137945aa66fb6f303c9536c6c" exitCode=0 Jul 07 12:48:22.750856 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:22.750462 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" event={"ID":"a44e2e6b-06a6-4cab-b490-783fdab05203","Type":"ContainerDied","Data":"c2ca979638c802843fbc20aeba93b9ac655c91e137945aa66fb6f303c9536c6c"} Jul 07 12:48:23.756655 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:23.756618 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" event={"ID":"a44e2e6b-06a6-4cab-b490-783fdab05203","Type":"ContainerStarted","Data":"a7df8c60778d30aaeacdc029c2d734e27fefbb4b006ac985767f11df5966cce1"} Jul 07 12:48:23.757082 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:23.756728 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:23.799337 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:23.799284 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" podStartSLOduration=3.18075091 podStartE2EDuration="7.799266539s" totalImagesPullingTime="3.608730715s" totalInitContainerRuntime="1.009784914s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:23.798321619 +0000 UTC m=+377.142975804" watchObservedRunningTime="2026-07-07 12:48:23.799266539 +0000 UTC m=+377.143920722" Jul 07 12:48:34.221368 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.221331 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg"] Jul 07 12:48:34.224663 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.224643 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.228605 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.228579 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 12:48:34.228751 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.228657 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 12:48:34.229491 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.229474 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 12:48:34.229557 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.229503 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 12:48:34.229852 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.229837 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hhbbh\"" Jul 07 12:48:34.229892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.229880 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 12:48:34.249913 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.249882 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg"] Jul 07 12:48:34.257107 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.257073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/465fb2a0-a462-4f50-a903-6ea31e46c4e7-cert\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.257270 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.257134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/465fb2a0-a462-4f50-a903-6ea31e46c4e7-manager-config\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.257270 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.257170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/465fb2a0-a462-4f50-a903-6ea31e46c4e7-metrics-cert\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.257270 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.257204 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tjzm\" (UniqueName: \"kubernetes.io/projected/465fb2a0-a462-4f50-a903-6ea31e46c4e7-kube-api-access-6tjzm\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.358337 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.358299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/465fb2a0-a462-4f50-a903-6ea31e46c4e7-cert\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.358550 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.358356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/465fb2a0-a462-4f50-a903-6ea31e46c4e7-manager-config\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.358550 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.358383 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/465fb2a0-a462-4f50-a903-6ea31e46c4e7-metrics-cert\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.358550 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.358409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tjzm\" (UniqueName: \"kubernetes.io/projected/465fb2a0-a462-4f50-a903-6ea31e46c4e7-kube-api-access-6tjzm\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.359141 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.359107 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/465fb2a0-a462-4f50-a903-6ea31e46c4e7-manager-config\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.360809 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.360793 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/465fb2a0-a462-4f50-a903-6ea31e46c4e7-metrics-cert\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.360912 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.360893 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/465fb2a0-a462-4f50-a903-6ea31e46c4e7-cert\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.371925 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.371893 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tjzm\" (UniqueName: \"kubernetes.io/projected/465fb2a0-a462-4f50-a903-6ea31e46c4e7-kube-api-access-6tjzm\") pod \"lws-controller-manager-65fd447f77-hsmvg\" (UID: \"465fb2a0-a462-4f50-a903-6ea31e46c4e7\") " pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.534032 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.533939 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:34.707453 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.707428 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg"] Jul 07 12:48:34.710062 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:48:34.710002 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465fb2a0_a462_4f50_a903_6ea31e46c4e7.slice/crio-c19c3eb43efbb792d4fe19019c27f5b0b4a4cb9eca8a58432d8efcee03161160 WatchSource:0}: Error finding container c19c3eb43efbb792d4fe19019c27f5b0b4a4cb9eca8a58432d8efcee03161160: Status 404 returned error can't find the container with id c19c3eb43efbb792d4fe19019c27f5b0b4a4cb9eca8a58432d8efcee03161160 Jul 07 12:48:34.763335 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.763300 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68cfd86456-rgm4b" Jul 07 12:48:34.801785 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:34.801687 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" event={"ID":"465fb2a0-a462-4f50-a903-6ea31e46c4e7","Type":"ContainerStarted","Data":"c19c3eb43efbb792d4fe19019c27f5b0b4a4cb9eca8a58432d8efcee03161160"} Jul 07 12:48:37.817994 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:37.817959 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" event={"ID":"465fb2a0-a462-4f50-a903-6ea31e46c4e7","Type":"ContainerStarted","Data":"cd1c624006af2e71d92a607cba4e0a3333dd95aa5bc9df7344ff92dc8d5de2fe"} Jul 07 12:48:37.818485 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:37.818081 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:37.864397 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:37.864339 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" podStartSLOduration=1.288384 podStartE2EDuration="3.86432298s" totalImagesPullingTime="2.57593898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:37.860169123 +0000 UTC m=+391.204823309" watchObservedRunningTime="2026-07-07 12:48:37.86432298 +0000 UTC m=+391.208977164" Jul 07 12:48:48.122167 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.122129 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5"] Jul 07 12:48:48.125306 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.125287 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.133340 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:48:48.133308 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" is forbidden: User \"system:node:ip-10-0-141-76.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-76.ec2.internal' and this object" podUID="bbf978ee-cbd1-4d1b-af76-e2045dd92a32" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.133340 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:48:48.133320 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"data-science-gateway-data-science-gateway-class-dockercfg-r595s\" is forbidden: User \"system:node:ip-10-0-141-76.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-76.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-r595s\"" type="*v1.Secret" Jul 07 12:48:48.133503 ip-10-0-141-76 kubenswrapper[2578]: E0707 12:48:48.133382 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-141-76.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-76.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap" Jul 07 12:48:48.173920 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.173874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.173920 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.173913 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.174214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.173938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.174214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.173967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.174214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.174028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.174214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.174075 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.174214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.174097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.174214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.174123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.174214 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.174176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qdt2\" (UniqueName: \"kubernetes.io/projected/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-kube-api-access-4qdt2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275522 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275522 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275619 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qdt2\" (UniqueName: \"kubernetes.io/projected/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-kube-api-access-4qdt2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275658 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275685 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.275793 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.275735 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.276154 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.276032 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.276154 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.276094 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.276154 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.276104 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.276322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.276264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.278157 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.278126 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.278374 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.278356 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.288026 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.287996 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.306014 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.305986 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qdt2\" (UniqueName: \"kubernetes.io/projected/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-kube-api-access-4qdt2\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:48.328892 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.328853 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5"] Jul 07 12:48:48.825915 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:48.825886 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65fd447f77-hsmvg" Jul 07 12:48:49.071629 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.071592 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 12:48:49.076267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.076215 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf978ee-cbd1-4d1b-af76-e2045dd92a32-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45vvd5\" (UID: \"bbf978ee-cbd1-4d1b-af76-e2045dd92a32\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:49.172834 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.172804 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-r595s\"" Jul 07 12:48:49.335563 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.335472 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:49.486498 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.486462 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5"] Jul 07 12:48:49.493384 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:48:49.493354 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf978ee_cbd1_4d1b_af76_e2045dd92a32.slice/crio-695750960a716383728ebd78a4e936af02208318cdfd2a6783bdefbc72e3e215 WatchSource:0}: Error finding container 695750960a716383728ebd78a4e936af02208318cdfd2a6783bdefbc72e3e215: Status 404 returned error can't find the container with id 695750960a716383728ebd78a4e936af02208318cdfd2a6783bdefbc72e3e215 Jul 07 12:48:49.570243 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.570211 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp"] Jul 07 12:48:49.575325 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.575304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.581694 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.581658 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 12:48:49.581694 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.581658 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 12:48:49.581991 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.581903 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 12:48:49.583224 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.583204 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-964q8\"" Jul 07 12:48:49.594075 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.593970 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp"] Jul 07 12:48:49.688351 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.688315 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4becf08f-fc61-4413-b9da-3ce0d7c91df8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.688351 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.688352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.688569 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.688407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.688569 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.688480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.688569 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.688561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.688669 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.688599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.688669 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.688649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vgtl\" (UniqueName: \"kubernetes.io/projected/4becf08f-fc61-4413-b9da-3ce0d7c91df8-kube-api-access-8vgtl\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.789322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.789286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4becf08f-fc61-4413-b9da-3ce0d7c91df8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.789322 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.789333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.789584 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.789369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.789584 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.789390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.789584 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.789455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.789584 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.789492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.789584 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.789542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vgtl\" (UniqueName: \"kubernetes.io/projected/4becf08f-fc61-4413-b9da-3ce0d7c91df8-kube-api-access-8vgtl\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.790400 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.790368 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.791767 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.791739 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4becf08f-fc61-4413-b9da-3ce0d7c91df8-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.792150 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.792112 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.792246 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.792128 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.792246 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.792204 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.801277 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.801252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4becf08f-fc61-4413-b9da-3ce0d7c91df8-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.801746 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.801718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vgtl\" (UniqueName: \"kubernetes.io/projected/4becf08f-fc61-4413-b9da-3ce0d7c91df8-kube-api-access-8vgtl\") pod \"istiod-openshift-gateway-94bb8fbfd-bnjzp\" (UID: \"4becf08f-fc61-4413-b9da-3ce0d7c91df8\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:49.870759 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.870657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" event={"ID":"bbf978ee-cbd1-4d1b-af76-e2045dd92a32","Type":"ContainerStarted","Data":"695750960a716383728ebd78a4e936af02208318cdfd2a6783bdefbc72e3e215"} Jul 07 12:48:49.884985 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:49.884954 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:50.032681 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:50.032651 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp"] Jul 07 12:48:50.035320 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:48:50.035288 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4becf08f_fc61_4413_b9da_3ce0d7c91df8.slice/crio-2dca08800cc2a2389d6eb1bd2a730d9b2f7a106edee6bc1e8dd087e64e833c70 WatchSource:0}: Error finding container 2dca08800cc2a2389d6eb1bd2a730d9b2f7a106edee6bc1e8dd087e64e833c70: Status 404 returned error can't find the container with id 2dca08800cc2a2389d6eb1bd2a730d9b2f7a106edee6bc1e8dd087e64e833c70 Jul 07 12:48:50.878681 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:50.878635 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" event={"ID":"4becf08f-fc61-4413-b9da-3ce0d7c91df8","Type":"ContainerStarted","Data":"2dca08800cc2a2389d6eb1bd2a730d9b2f7a106edee6bc1e8dd087e64e833c70"} Jul 07 12:48:53.278632 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.278574 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 12:48:53.278948 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.278692 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 12:48:53.278948 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.278739 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 12:48:53.279018 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.278961 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 12:48:53.279092 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.279024 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 12:48:53.894505 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.894457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" event={"ID":"4becf08f-fc61-4413-b9da-3ce0d7c91df8","Type":"ContainerStarted","Data":"b7c8c05a78cf272f0354b463ee3f604abb59750782d96778ff37c8e227ffca84"} Jul 07 12:48:53.894705 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.894544 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:48:53.896305 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.896277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" event={"ID":"bbf978ee-cbd1-4d1b-af76-e2045dd92a32","Type":"ContainerStarted","Data":"14ea5e90a300c92f5c2ac71336035c97bc18b588f8d1306fbcaf631f82004da1"} Jul 07 12:48:53.990408 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:53.990344 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" podStartSLOduration=1.748925625 podStartE2EDuration="4.990325492s" totalImagesPullingTime="3.241399867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:53.987448569 +0000 UTC m=+407.332102754" watchObservedRunningTime="2026-07-07 12:48:53.990325492 +0000 UTC m=+407.334979677" Jul 07 12:48:54.074869 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:54.074813 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" podStartSLOduration=2.291956067 podStartE2EDuration="6.074796293s" totalImagesPullingTime="3.782840226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:48:54.074326605 +0000 UTC m=+407.418980790" watchObservedRunningTime="2026-07-07 12:48:54.074796293 +0000 UTC m=+407.419450545" Jul 07 12:48:54.336134 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:54.336094 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:54.343092 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:54.343061 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:54.900912 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:54.900879 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:54.901745 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:54.901725 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45vvd5" Jul 07 12:48:54.902871 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:48:54.902852 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bnjzp" Jul 07 12:52:07.044787 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:07.044759 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:52:07.048877 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:07.048855 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:52:34.577987 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.577907 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn"] Jul 07 12:52:34.581302 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.581282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.584259 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.584237 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 12:52:34.585267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.585244 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 12:52:34.585267 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.585259 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 12:52:34.585436 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.585255 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f7d92\"" Jul 07 12:52:34.589825 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.589798 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn"] Jul 07 12:52:34.677584 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.677543 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.677762 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.677616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.677762 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.677637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.677762 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.677663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2e12c7-a853-4efa-a3b0-00a02147ff11-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.677869 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.677776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.677869 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.677807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvcvj\" (UniqueName: \"kubernetes.io/projected/ac2e12c7-a853-4efa-a3b0-00a02147ff11-kube-api-access-hvcvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.778460 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.778425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.778651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.778480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.778651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.778505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.778651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.778542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2e12c7-a853-4efa-a3b0-00a02147ff11-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.778651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.778598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.778651 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.778622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvcvj\" (UniqueName: \"kubernetes.io/projected/ac2e12c7-a853-4efa-a3b0-00a02147ff11-kube-api-access-hvcvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.779001 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.778954 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.779140 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.779009 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.779140 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.779032 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.780802 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.780781 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac2e12c7-a853-4efa-a3b0-00a02147ff11-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.781207 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.781187 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac2e12c7-a853-4efa-a3b0-00a02147ff11-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.786879 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.786850 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvcvj\" (UniqueName: \"kubernetes.io/projected/ac2e12c7-a853-4efa-a3b0-00a02147ff11-kube-api-access-hvcvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn\" (UID: \"ac2e12c7-a853-4efa-a3b0-00a02147ff11\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:34.892013 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:34.891912 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:35.024620 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:35.024559 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn"] Jul 07 12:52:35.026859 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:52:35.026833 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2e12c7_a853_4efa_a3b0_00a02147ff11.slice/crio-aef6fb5388e2cbf1f4b177340918056a84981ba642faabeda10ad2abd078ae1f WatchSource:0}: Error finding container aef6fb5388e2cbf1f4b177340918056a84981ba642faabeda10ad2abd078ae1f: Status 404 returned error can't find the container with id aef6fb5388e2cbf1f4b177340918056a84981ba642faabeda10ad2abd078ae1f Jul 07 12:52:35.838946 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:35.838911 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" event={"ID":"ac2e12c7-a853-4efa-a3b0-00a02147ff11","Type":"ContainerStarted","Data":"aef6fb5388e2cbf1f4b177340918056a84981ba642faabeda10ad2abd078ae1f"} Jul 07 12:52:36.844152 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:36.844112 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" event={"ID":"ac2e12c7-a853-4efa-a3b0-00a02147ff11","Type":"ContainerStarted","Data":"de4df6440f885f3967aadbf31272a321a8da7e0bbf5d35b3d79a450ea37f6404"} Jul 07 12:52:36.864297 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:36.864243 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" podStartSLOduration=1.685665629 podStartE2EDuration="2.864227054s" totalImagesPullingTime="1.178561425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:52:36.864136739 +0000 UTC m=+630.208790923" watchObservedRunningTime="2026-07-07 12:52:36.864227054 +0000 UTC m=+630.208881238" Jul 07 12:52:44.893087 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:44.893014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:44.895573 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:44.893114 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:44.905712 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:44.905690 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:45.893905 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:45.893874 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn" Jul 07 12:52:59.467256 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.467222 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z"] Jul 07 12:52:59.470587 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.470564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.473272 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.473250 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 12:52:59.483142 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.483110 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z"] Jul 07 12:52:59.592215 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.592172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.592401 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.592227 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.592401 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.592294 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xvq9\" (UniqueName: \"kubernetes.io/projected/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-kube-api-access-7xvq9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.592401 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.592332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.592401 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.592368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.592538 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.592435 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.693587 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.693540 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.693797 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.693600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.693797 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.693669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.693797 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.693706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.693797 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.693738 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xvq9\" (UniqueName: \"kubernetes.io/projected/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-kube-api-access-7xvq9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.693797 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.693769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.694095 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.693945 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.694095 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.694002 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.694233 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.694212 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.696103 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.696081 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.696422 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.696399 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.703347 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.703321 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xvq9\" (UniqueName: \"kubernetes.io/projected/3d5fb75f-f0d2-4bc8-8661-39e13d32923c-kube-api-access-7xvq9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z\" (UID: \"3d5fb75f-f0d2-4bc8-8661-39e13d32923c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.781387 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.781280 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:52:59.915728 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.915701 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z"] Jul 07 12:52:59.917818 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:52:59.917781 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5fb75f_f0d2_4bc8_8661_39e13d32923c.slice/crio-c412fbe286b91eef12edc8b2bea0e0489438a300763d99d717c5689440579e2f WatchSource:0}: Error finding container c412fbe286b91eef12edc8b2bea0e0489438a300763d99d717c5689440579e2f: Status 404 returned error can't find the container with id c412fbe286b91eef12edc8b2bea0e0489438a300763d99d717c5689440579e2f Jul 07 12:52:59.919613 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.919587 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 12:52:59.945111 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:52:59.945076 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" event={"ID":"3d5fb75f-f0d2-4bc8-8661-39e13d32923c","Type":"ContainerStarted","Data":"c412fbe286b91eef12edc8b2bea0e0489438a300763d99d717c5689440579e2f"} Jul 07 12:53:01.955750 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:01.955707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" event={"ID":"3d5fb75f-f0d2-4bc8-8661-39e13d32923c","Type":"ContainerStarted","Data":"46c6c2ae25ad3d83d26198c4529e972e43c662c4068b174a84225cd1cc2b1920"} Jul 07 12:53:01.975716 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:01.975661 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" podStartSLOduration=1.580528216 podStartE2EDuration="2.975646707s" totalImagesPullingTime="1.395118491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:53:01.974472609 +0000 UTC m=+655.319126794" watchObservedRunningTime="2026-07-07 12:53:01.975646707 +0000 UTC m=+655.320300890" Jul 07 12:53:08.908619 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:08.908578 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9"] Jul 07 12:53:08.914489 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:08.914465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:08.921139 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:08.921110 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 12:53:08.933829 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:08.933800 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9"] Jul 07 12:53:09.085291 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.085250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.085291 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.085297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.085530 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.085352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.085530 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.085399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hstv9\" (UniqueName: \"kubernetes.io/projected/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-kube-api-access-hstv9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.085530 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.085424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.085530 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.085446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186330 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186232 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hstv9\" (UniqueName: \"kubernetes.io/projected/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-kube-api-access-hstv9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186330 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186330 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186620 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186620 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186620 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186834 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186810 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186874 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186848 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.186920 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.186893 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.188757 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.188731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.188991 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.188974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.195890 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.195863 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hstv9\" (UniqueName: \"kubernetes.io/projected/9df3024b-e63c-4bbb-95b2-d7fb3f309f42-kube-api-access-hstv9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w28b9\" (UID: \"9df3024b-e63c-4bbb-95b2-d7fb3f309f42\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.224777 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.224740 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:09.361308 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.361239 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9"] Jul 07 12:53:09.363522 ip-10-0-141-76 kubenswrapper[2578]: W0707 12:53:09.363496 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df3024b_e63c_4bbb_95b2_d7fb3f309f42.slice/crio-e45f1da078adaa3b6aa029b46cccd83e01105e8dc3d9cf03d6d5154634075f14 WatchSource:0}: Error finding container e45f1da078adaa3b6aa029b46cccd83e01105e8dc3d9cf03d6d5154634075f14: Status 404 returned error can't find the container with id e45f1da078adaa3b6aa029b46cccd83e01105e8dc3d9cf03d6d5154634075f14 Jul 07 12:53:09.782219 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.782180 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:53:09.782404 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.782253 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:53:09.794877 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.794849 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:53:09.992272 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.992183 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" event={"ID":"9df3024b-e63c-4bbb-95b2-d7fb3f309f42","Type":"ContainerStarted","Data":"448dc198a83e541c33b02314819b09265bbcbbed0825aacc96a0915acf9ccc9f"} Jul 07 12:53:09.992272 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:09.992243 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" event={"ID":"9df3024b-e63c-4bbb-95b2-d7fb3f309f42","Type":"ContainerStarted","Data":"e45f1da078adaa3b6aa029b46cccd83e01105e8dc3d9cf03d6d5154634075f14"} Jul 07 12:53:10.003583 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:10.003552 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z" Jul 07 12:53:10.011143 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:10.011089 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" podStartSLOduration=1.800352731 podStartE2EDuration="2.01107408s" totalImagesPullingTime="210.721349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:53:10.010074663 +0000 UTC m=+663.354728848" watchObservedRunningTime="2026-07-07 12:53:10.01107408 +0000 UTC m=+663.355728264" Jul 07 12:53:19.225249 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:19.225174 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:19.225702 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:19.225286 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:19.237778 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:19.237749 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:53:20.046358 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:53:20.046328 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w28b9" Jul 07 12:57:07.071849 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:57:07.071767 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 12:57:07.074810 ip-10-0-141-76 kubenswrapper[2578]: I0707 12:57:07.074782 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:00:00.146171 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.146089 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5"] Jul 07 13:00:00.149285 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.149270 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" Jul 07 13:00:00.152154 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.152134 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fwnbg\"" Jul 07 13:00:00.152154 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.152145 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 07 13:00:00.152320 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.152204 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 07 13:00:00.165726 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.165701 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5"] Jul 07 13:00:00.318676 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.318637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrqxd\" (UniqueName: \"kubernetes.io/projected/fd2b4488-bfab-4cc6-b15d-3cac78e35746-kube-api-access-rrqxd\") pod \"maas-api-key-cleanup-29723820-52sf5\" (UID: \"fd2b4488-bfab-4cc6-b15d-3cac78e35746\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" Jul 07 13:00:00.420231 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.420128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrqxd\" (UniqueName: \"kubernetes.io/projected/fd2b4488-bfab-4cc6-b15d-3cac78e35746-kube-api-access-rrqxd\") pod \"maas-api-key-cleanup-29723820-52sf5\" (UID: \"fd2b4488-bfab-4cc6-b15d-3cac78e35746\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" Jul 07 13:00:00.429397 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.429366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrqxd\" (UniqueName: \"kubernetes.io/projected/fd2b4488-bfab-4cc6-b15d-3cac78e35746-kube-api-access-rrqxd\") pod \"maas-api-key-cleanup-29723820-52sf5\" (UID: \"fd2b4488-bfab-4cc6-b15d-3cac78e35746\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" Jul 07 13:00:00.459816 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.459773 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" Jul 07 13:00:00.592574 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.592534 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5"] Jul 07 13:00:00.595684 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:00:00.595657 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2b4488_bfab_4cc6_b15d_3cac78e35746.slice/crio-bc9a9b0a6b2c8e1b162a7d58e039d64b817cf7460a67358af86cb644cb065457 WatchSource:0}: Error finding container bc9a9b0a6b2c8e1b162a7d58e039d64b817cf7460a67358af86cb644cb065457: Status 404 returned error can't find the container with id bc9a9b0a6b2c8e1b162a7d58e039d64b817cf7460a67358af86cb644cb065457 Jul 07 13:00:00.597929 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.597910 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:00:00.812159 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:00.812067 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" event={"ID":"fd2b4488-bfab-4cc6-b15d-3cac78e35746","Type":"ContainerStarted","Data":"bc9a9b0a6b2c8e1b162a7d58e039d64b817cf7460a67358af86cb644cb065457"} Jul 07 13:00:19.900301 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:19.900206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" event={"ID":"fd2b4488-bfab-4cc6-b15d-3cac78e35746","Type":"ContainerStarted","Data":"eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574"} Jul 07 13:00:19.919331 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:00:19.919281 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" podStartSLOduration=1.157593496 podStartE2EDuration="19.91926267s" totalImagesPullingTime="18.761669174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:00:19.91627198 +0000 UTC m=+1093.260926167" watchObservedRunningTime="2026-07-07 13:00:19.91926267 +0000 UTC m=+1093.263916857" Jul 07 13:01:00.011950 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:00.011900 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5"] Jul 07 13:01:00.012581 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:00.012179 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" podUID="fd2b4488-bfab-4cc6-b15d-3cac78e35746" containerName="cleanup" containerID="cri-o://eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574" gracePeriod=30 Jul 07 13:01:30.208887 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.208860 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" Jul 07 13:01:30.217715 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.217685 2578 generic.go:356] "Generic (PLEG): container finished" podID="fd2b4488-bfab-4cc6-b15d-3cac78e35746" containerID="eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574" exitCode=137 Jul 07 13:01:30.217864 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.217729 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" Jul 07 13:01:30.217864 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.217725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" event={"ID":"fd2b4488-bfab-4cc6-b15d-3cac78e35746","Type":"ContainerDied","Data":"eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574"} Jul 07 13:01:30.217864 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.217845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5" event={"ID":"fd2b4488-bfab-4cc6-b15d-3cac78e35746","Type":"ContainerDied","Data":"bc9a9b0a6b2c8e1b162a7d58e039d64b817cf7460a67358af86cb644cb065457"} Jul 07 13:01:30.217983 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.217869 2578 scope.go:117] "RemoveContainer" containerID="eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574" Jul 07 13:01:30.225972 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.225955 2578 scope.go:117] "RemoveContainer" containerID="eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574" Jul 07 13:01:30.226255 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:01:30.226235 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574\": container with ID starting with eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574 not found: ID does not exist" containerID="eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574" Jul 07 13:01:30.226331 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.226269 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574"} err="failed to get container status \"eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574\": rpc error: code = NotFound desc = could not find container \"eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574\": container with ID starting with eeaf7b66ca58555f300a5f9226b4c358ca82539936d04f63207d6f88c803b574 not found: ID does not exist" Jul 07 13:01:30.293207 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.293118 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrqxd\" (UniqueName: \"kubernetes.io/projected/fd2b4488-bfab-4cc6-b15d-3cac78e35746-kube-api-access-rrqxd\") pod \"fd2b4488-bfab-4cc6-b15d-3cac78e35746\" (UID: \"fd2b4488-bfab-4cc6-b15d-3cac78e35746\") " Jul 07 13:01:30.295432 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.295395 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2b4488-bfab-4cc6-b15d-3cac78e35746-kube-api-access-rrqxd" (OuterVolumeSpecName: "kube-api-access-rrqxd") pod "fd2b4488-bfab-4cc6-b15d-3cac78e35746" (UID: "fd2b4488-bfab-4cc6-b15d-3cac78e35746"). InnerVolumeSpecName "kube-api-access-rrqxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:01:30.394670 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.394634 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrqxd\" (UniqueName: \"kubernetes.io/projected/fd2b4488-bfab-4cc6-b15d-3cac78e35746-kube-api-access-rrqxd\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:01:30.541868 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.541832 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5"] Jul 07 13:01:30.547465 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:30.547388 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723820-52sf5"] Jul 07 13:01:31.156699 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:01:31.156666 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd2b4488-bfab-4cc6-b15d-3cac78e35746" path="/var/lib/kubelet/pods/fd2b4488-bfab-4cc6-b15d-3cac78e35746/volumes" Jul 07 13:02:07.105056 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:02:07.105023 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:02:07.105601 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:02:07.105023 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:07:07.129848 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:07.129817 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:07:07.132774 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:07.132752 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:07:24.240085 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.240052 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24"] Jul 07 13:07:24.240523 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.240399 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd2b4488-bfab-4cc6-b15d-3cac78e35746" containerName="cleanup" Jul 07 13:07:24.240523 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.240410 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd2b4488-bfab-4cc6-b15d-3cac78e35746" containerName="cleanup" Jul 07 13:07:24.240523 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.240469 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd2b4488-bfab-4cc6-b15d-3cac78e35746" containerName="cleanup" Jul 07 13:07:24.243264 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.243243 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.246281 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.246260 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-abdf58a0-openshift-default-dockercfg-5md2z\"" Jul 07 13:07:24.270321 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.270291 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24"] Jul 07 13:07:24.359379 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-socket\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-data\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq9vh\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-kube-api-access-hq9vh\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-credential-socket\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-podinfo\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd6408f6-f130-4952-b83c-a14b423cbf78-istiod-ca-cert\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359759 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-certs\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359759 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-envoy\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.359759 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.359647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-token\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460147 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq9vh\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-kube-api-access-hq9vh\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-credential-socket\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460209 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-podinfo\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd6408f6-f130-4952-b83c-a14b423cbf78-istiod-ca-cert\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-certs\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-envoy\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-token\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460651 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-socket\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460651 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-data\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460722 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460683 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-credential-socket\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460722 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460706 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-certs\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.460956 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.460934 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-socket\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.461082 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.461062 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-data\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.461187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.461163 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd6408f6-f130-4952-b83c-a14b423cbf78-istiod-ca-cert\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.462781 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.462762 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-envoy\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.462877 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.462829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-podinfo\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.470944 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.470921 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-token\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.471233 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.471212 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq9vh\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-kube-api-access-hq9vh\") pod \"e2e-ait-abdf58a0-openshift-default-c856c557-rnm24\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.557000 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.556908 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:24.695353 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.695323 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24"] Jul 07 13:07:24.697613 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:07:24.697581 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd6408f6_f130_4952_b83c_a14b423cbf78.slice/crio-da491647aa38119dc97ba74c9971d4c4cf28421635deb3a080d604ce0e6a8bd2 WatchSource:0}: Error finding container da491647aa38119dc97ba74c9971d4c4cf28421635deb3a080d604ce0e6a8bd2: Status 404 returned error can't find the container with id da491647aa38119dc97ba74c9971d4c4cf28421635deb3a080d604ce0e6a8bd2 Jul 07 13:07:24.699680 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.699647 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:24.699839 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.699722 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:24.699839 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.699764 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:24.812197 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.812100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" event={"ID":"cd6408f6-f130-4952-b83c-a14b423cbf78","Type":"ContainerStarted","Data":"db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f"} Jul 07 13:07:24.812197 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.812168 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" event={"ID":"cd6408f6-f130-4952-b83c-a14b423cbf78","Type":"ContainerStarted","Data":"da491647aa38119dc97ba74c9971d4c4cf28421635deb3a080d604ce0e6a8bd2"} Jul 07 13:07:24.852558 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:24.852503 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" podStartSLOduration=0.852485335 podStartE2EDuration="852.485335ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:07:24.851183707 +0000 UTC m=+1518.195837891" watchObservedRunningTime="2026-07-07 13:07:24.852485335 +0000 UTC m=+1518.197139571" Jul 07 13:07:25.558147 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:25.558109 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:25.562820 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:25.562791 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:25.816821 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:25.816735 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:25.817641 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:25.817620 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:30.677488 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:30.677399 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24"] Jul 07 13:07:30.677860 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:30.677603 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" podUID="cd6408f6-f130-4952-b83c-a14b423cbf78" containerName="istio-proxy" containerID="cri-o://db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f" gracePeriod=30 Jul 07 13:07:36.883556 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.883513 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7"] Jul 07 13:07:36.887930 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.887902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.891052 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.891018 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a290d93f-openshift-default-dockercfg-nzjc2\"" Jul 07 13:07:36.901824 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.901798 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7"] Jul 07 13:07:36.997583 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-socket\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997583 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbc5l\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-kube-api-access-cbc5l\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997610 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-certs\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afa59c-32fe-45d0-89ad-d3321dc97cca-istiod-ca-cert\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-data\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-token\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997821 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-envoy\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-credential-socket\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:36.997965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:36.997875 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-podinfo\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-envoy\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-credential-socket\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098877 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-podinfo\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098877 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098708 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-socket\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098877 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbc5l\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-kube-api-access-cbc5l\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098877 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-certs\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098877 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098809 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afa59c-32fe-45d0-89ad-d3321dc97cca-istiod-ca-cert\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.098877 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-data\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.099220 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.098907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-token\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.099220 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.099113 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-credential-socket\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.099220 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.099203 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-socket\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.099452 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.099430 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-certs\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.099507 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.099464 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-data\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.099719 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.099688 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afa59c-32fe-45d0-89ad-d3321dc97cca-istiod-ca-cert\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.101195 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.101164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-envoy\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.101601 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.101580 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-podinfo\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.107643 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.107616 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-token\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.107734 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.107669 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbc5l\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-kube-api-access-cbc5l\") pod \"e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.206353 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.206267 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:37.365785 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.365757 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7"] Jul 07 13:07:37.367191 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:07:37.367163 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29afa59c_32fe_45d0_89ad_d3321dc97cca.slice/crio-587c7b2fd8ea8d76822c26939fc8b654d968998849667f11a882ce0cbd0c8a5a WatchSource:0}: Error finding container 587c7b2fd8ea8d76822c26939fc8b654d968998849667f11a882ce0cbd0c8a5a: Status 404 returned error can't find the container with id 587c7b2fd8ea8d76822c26939fc8b654d968998849667f11a882ce0cbd0c8a5a Jul 07 13:07:37.370092 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.370058 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:37.370179 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.370129 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:37.370179 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.370157 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:37.530990 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.530965 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:37.601509 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601470 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-data\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601509 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601522 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-certs\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601777 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601555 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-envoy\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601777 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601619 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd6408f6-f130-4952-b83c-a14b423cbf78-istiod-ca-cert\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601777 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601649 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-credential-socket\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601777 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601765 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-socket\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601812 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq9vh\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-kube-api-access-hq9vh\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601861 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-token\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.601981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.601896 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-podinfo\") pod \"cd6408f6-f130-4952-b83c-a14b423cbf78\" (UID: \"cd6408f6-f130-4952-b83c-a14b423cbf78\") " Jul 07 13:07:37.602167 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.602114 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:37.602223 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.602170 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd6408f6-f130-4952-b83c-a14b423cbf78-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:07:37.602298 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.602277 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.602354 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.602300 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd6408f6-f130-4952-b83c-a14b423cbf78-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.602408 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.602375 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:37.602505 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.602482 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:37.602639 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.602618 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:37.604241 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.604218 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:37.604473 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.604452 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-kube-api-access-hq9vh" (OuterVolumeSpecName: "kube-api-access-hq9vh") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "kube-api-access-hq9vh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:07:37.604567 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.604545 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:07:37.604747 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.604728 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd6408f6-f130-4952-b83c-a14b423cbf78" (UID: "cd6408f6-f130-4952-b83c-a14b423cbf78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:07:37.703550 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.703508 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.703550 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.703544 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq9vh\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-kube-api-access-hq9vh\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.703550 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.703557 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.703797 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.703571 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.703797 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.703582 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.703797 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.703596 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.703797 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.703606 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd6408f6-f130-4952-b83c-a14b423cbf78-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:37.880985 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.880942 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" event={"ID":"29afa59c-32fe-45d0-89ad-d3321dc97cca","Type":"ContainerStarted","Data":"70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b"} Jul 07 13:07:37.881173 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.881059 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" event={"ID":"29afa59c-32fe-45d0-89ad-d3321dc97cca","Type":"ContainerStarted","Data":"587c7b2fd8ea8d76822c26939fc8b654d968998849667f11a882ce0cbd0c8a5a"} Jul 07 13:07:37.882275 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.882250 2578 generic.go:356] "Generic (PLEG): container finished" podID="cd6408f6-f130-4952-b83c-a14b423cbf78" containerID="db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f" exitCode=0 Jul 07 13:07:37.882371 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.882302 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" Jul 07 13:07:37.882371 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.882327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" event={"ID":"cd6408f6-f130-4952-b83c-a14b423cbf78","Type":"ContainerDied","Data":"db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f"} Jul 07 13:07:37.882455 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.882394 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24" event={"ID":"cd6408f6-f130-4952-b83c-a14b423cbf78","Type":"ContainerDied","Data":"da491647aa38119dc97ba74c9971d4c4cf28421635deb3a080d604ce0e6a8bd2"} Jul 07 13:07:37.882455 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.882433 2578 scope.go:117] "RemoveContainer" containerID="db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f" Jul 07 13:07:37.891694 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.891677 2578 scope.go:117] "RemoveContainer" containerID="db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f" Jul 07 13:07:37.892014 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:07:37.891992 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f\": container with ID starting with db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f not found: ID does not exist" containerID="db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f" Jul 07 13:07:37.892093 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.892019 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f"} err="failed to get container status \"db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f\": rpc error: code = NotFound desc = could not find container \"db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f\": container with ID starting with db3e83059ef695903cf6ae14f84aa215175f08159cfd6c2de555a70bff5f645f not found: ID does not exist" Jul 07 13:07:37.902507 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.902460 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" podStartSLOduration=1.9024441520000002 podStartE2EDuration="1.902444152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:07:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:07:37.900824773 +0000 UTC m=+1531.245478958" watchObservedRunningTime="2026-07-07 13:07:37.902444152 +0000 UTC m=+1531.247098422" Jul 07 13:07:37.924331 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.924249 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24"] Jul 07 13:07:37.925784 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:37.925761 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-abdf58a0-openshift-default-c856c557-rnm24"] Jul 07 13:07:38.207248 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:38.207156 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:38.212320 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:38.212293 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:38.888921 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:38.888888 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:38.889941 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:38.889924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:39.157112 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:39.157003 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd6408f6-f130-4952-b83c-a14b423cbf78" path="/var/lib/kubelet/pods/cd6408f6-f130-4952-b83c-a14b423cbf78/volumes" Jul 07 13:07:49.243549 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:49.243515 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7"] Jul 07 13:07:49.244002 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:49.243754 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" podUID="29afa59c-32fe-45d0-89ad-d3321dc97cca" containerName="istio-proxy" containerID="cri-o://70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b" gracePeriod=30 Jul 07 13:07:50.798028 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.798000 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:50.922018 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.921977 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbc5l\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-kube-api-access-cbc5l\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922018 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922025 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-podinfo\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922304 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922083 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-certs\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922304 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922142 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-envoy\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922304 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922198 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-socket\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922304 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922222 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afa59c-32fe-45d0-89ad-d3321dc97cca-istiod-ca-cert\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922304 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922267 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-credential-socket\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922304 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922297 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-data\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922598 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922374 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-token\") pod \"29afa59c-32fe-45d0-89ad-d3321dc97cca\" (UID: \"29afa59c-32fe-45d0-89ad-d3321dc97cca\") " Jul 07 13:07:50.922598 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922409 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:50.922598 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922512 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:50.922755 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922674 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:50.922817 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922758 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:50.922817 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922780 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:50.922908 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.922835 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29afa59c-32fe-45d0-89ad-d3321dc97cca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:07:50.923353 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.923328 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-data" (OuterVolumeSpecName: "istio-data") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:50.924424 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.924405 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-kube-api-access-cbc5l" (OuterVolumeSpecName: "kube-api-access-cbc5l") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "kube-api-access-cbc5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:07:50.924490 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.924453 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:07:50.924885 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.924868 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-token" (OuterVolumeSpecName: "istio-token") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:07:50.924933 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.924887 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29afa59c-32fe-45d0-89ad-d3321dc97cca" (UID: "29afa59c-32fe-45d0-89ad-d3321dc97cca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:07:50.946061 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.946008 2578 generic.go:356] "Generic (PLEG): container finished" podID="29afa59c-32fe-45d0-89ad-d3321dc97cca" containerID="70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b" exitCode=0 Jul 07 13:07:50.946186 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.946071 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" Jul 07 13:07:50.946186 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.946084 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" event={"ID":"29afa59c-32fe-45d0-89ad-d3321dc97cca","Type":"ContainerDied","Data":"70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b"} Jul 07 13:07:50.946186 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.946173 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7" event={"ID":"29afa59c-32fe-45d0-89ad-d3321dc97cca","Type":"ContainerDied","Data":"587c7b2fd8ea8d76822c26939fc8b654d968998849667f11a882ce0cbd0c8a5a"} Jul 07 13:07:50.946300 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.946205 2578 scope.go:117] "RemoveContainer" containerID="70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b" Jul 07 13:07:50.955188 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.955168 2578 scope.go:117] "RemoveContainer" containerID="70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b" Jul 07 13:07:50.955433 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:07:50.955411 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b\": container with ID starting with 70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b not found: ID does not exist" containerID="70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b" Jul 07 13:07:50.955480 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.955442 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b"} err="failed to get container status \"70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b\": rpc error: code = NotFound desc = could not find container \"70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b\": container with ID starting with 70961a9e77d352a5a66c4f78be1dcef254951ff51637cc3857f0b3a24fe9a65b not found: ID does not exist" Jul 07 13:07:50.967903 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.967872 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7"] Jul 07 13:07:50.971643 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:50.971620 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a290d93f-openshift-default-64785dcc55-ttlx7"] Jul 07 13:07:51.023661 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.023620 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbc5l\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-kube-api-access-cbc5l\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:51.023661 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.023654 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:51.023661 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.023663 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:51.023661 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.023671 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29afa59c-32fe-45d0-89ad-d3321dc97cca-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:51.023910 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.023679 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:51.023910 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.023687 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:51.023910 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.023695 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29afa59c-32fe-45d0-89ad-d3321dc97cca-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:07:51.156687 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:51.156594 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29afa59c-32fe-45d0-89ad-d3321dc97cca" path="/var/lib/kubelet/pods/29afa59c-32fe-45d0-89ad-d3321dc97cca/volumes" Jul 07 13:07:55.789011 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.788976 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj"] Jul 07 13:07:55.789511 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.789461 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6408f6-f130-4952-b83c-a14b423cbf78" containerName="istio-proxy" Jul 07 13:07:55.789511 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.789480 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6408f6-f130-4952-b83c-a14b423cbf78" containerName="istio-proxy" Jul 07 13:07:55.789511 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.789507 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29afa59c-32fe-45d0-89ad-d3321dc97cca" containerName="istio-proxy" Jul 07 13:07:55.789743 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.789515 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="29afa59c-32fe-45d0-89ad-d3321dc97cca" containerName="istio-proxy" Jul 07 13:07:55.789743 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.789605 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd6408f6-f130-4952-b83c-a14b423cbf78" containerName="istio-proxy" Jul 07 13:07:55.789743 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.789619 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="29afa59c-32fe-45d0-89ad-d3321dc97cca" containerName="istio-proxy" Jul 07 13:07:55.792844 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.792821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.795857 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.795828 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-087366f3-openshift-default-dockercfg-g45jg\"" Jul 07 13:07:55.807434 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.807405 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj"] Jul 07 13:07:55.874819 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.874781 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-data\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.874991 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.874837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-token\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.874991 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.874891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-certs\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.874991 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.874929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-socket\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.874991 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.874947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-credential-socket\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.874991 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.874967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istiod-ca-cert\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.875184 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.875009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-podinfo\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.875184 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.875092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-envoy\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.875184 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.875168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqdq6\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-kube-api-access-sqdq6\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.975930 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.975895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-certs\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976152 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.975944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-socket\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976152 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.975961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-credential-socket\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976152 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.975987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istiod-ca-cert\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976152 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-podinfo\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976152 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-envoy\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976152 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqdq6\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-kube-api-access-sqdq6\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976487 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-data\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976487 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-token\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976487 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976405 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-certs\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976487 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976427 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-socket\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976686 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-credential-socket\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976751 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976706 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-data\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.976866 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.976843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istiod-ca-cert\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.978714 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.978692 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-podinfo\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.978714 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.978703 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-envoy\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.984295 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.984271 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-token\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:55.984412 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:55.984397 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqdq6\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-kube-api-access-sqdq6\") pod \"e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:56.106243 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:56.106206 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:56.245417 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:56.245393 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj"] Jul 07 13:07:56.247085 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:07:56.247053 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151c5ffc_94ee_48bd_9853_a142ac8b8f79.slice/crio-ffb73c615a6f02fb24c7ecc33c58f92945c079c70b79a74fed94ab8209f37e22 WatchSource:0}: Error finding container ffb73c615a6f02fb24c7ecc33c58f92945c079c70b79a74fed94ab8209f37e22: Status 404 returned error can't find the container with id ffb73c615a6f02fb24c7ecc33c58f92945c079c70b79a74fed94ab8209f37e22 Jul 07 13:07:56.249322 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:56.249284 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:56.249400 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:56.249370 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:56.249439 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:56.249415 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:07:56.976382 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:56.976285 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" event={"ID":"151c5ffc-94ee-48bd-9853-a142ac8b8f79","Type":"ContainerStarted","Data":"74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff"} Jul 07 13:07:56.976382 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:56.976341 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" event={"ID":"151c5ffc-94ee-48bd-9853-a142ac8b8f79","Type":"ContainerStarted","Data":"ffb73c615a6f02fb24c7ecc33c58f92945c079c70b79a74fed94ab8209f37e22"} Jul 07 13:07:57.018825 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:57.018766 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" podStartSLOduration=2.018750009 podStartE2EDuration="2.018750009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:07:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:07:57.018118832 +0000 UTC m=+1550.362773015" watchObservedRunningTime="2026-07-07 13:07:57.018750009 +0000 UTC m=+1550.363404194" Jul 07 13:07:57.106842 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:57.106802 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:57.111819 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:57.111794 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:57.981812 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:57.981782 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:07:57.982918 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:07:57.982906 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:08:01.545181 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:01.545144 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj"] Jul 07 13:08:01.545591 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:01.545328 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" podUID="151c5ffc-94ee-48bd-9853-a142ac8b8f79" containerName="istio-proxy" containerID="cri-o://74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff" gracePeriod=30 Jul 07 13:08:06.806939 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.806873 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:08:06.974941 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.974904 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-envoy\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975148 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.974957 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-credential-socket\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975148 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.974981 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-socket\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975148 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975006 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-data\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975148 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975090 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-podinfo\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975148 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975123 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istiod-ca-cert\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975148 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975147 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqdq6\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-kube-api-access-sqdq6\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975458 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975193 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-token\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975458 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975252 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-certs\") pod \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\" (UID: \"151c5ffc-94ee-48bd-9853-a142ac8b8f79\") " Jul 07 13:08:06.975458 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975261 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:08:06.975458 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975337 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:08:06.975458 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975378 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-data" (OuterVolumeSpecName: "istio-data") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:08:06.975692 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975535 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:08:06.975692 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975595 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:06.975692 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975593 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:08:06.975692 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975610 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:06.975692 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.975620 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:06.977373 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.977347 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-token" (OuterVolumeSpecName: "istio-token") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:08:06.977373 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.977349 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:08:06.977643 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.977628 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:08:06.977756 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:06.977731 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-kube-api-access-sqdq6" (OuterVolumeSpecName: "kube-api-access-sqdq6") pod "151c5ffc-94ee-48bd-9853-a142ac8b8f79" (UID: "151c5ffc-94ee-48bd-9853-a142ac8b8f79"). InnerVolumeSpecName "kube-api-access-sqdq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:08:07.028400 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.028365 2578 generic.go:356] "Generic (PLEG): container finished" podID="151c5ffc-94ee-48bd-9853-a142ac8b8f79" containerID="74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff" exitCode=0 Jul 07 13:08:07.028551 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.028412 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" event={"ID":"151c5ffc-94ee-48bd-9853-a142ac8b8f79","Type":"ContainerDied","Data":"74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff"} Jul 07 13:08:07.028551 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.028428 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" Jul 07 13:08:07.028551 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.028455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj" event={"ID":"151c5ffc-94ee-48bd-9853-a142ac8b8f79","Type":"ContainerDied","Data":"ffb73c615a6f02fb24c7ecc33c58f92945c079c70b79a74fed94ab8209f37e22"} Jul 07 13:08:07.028551 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.028498 2578 scope.go:117] "RemoveContainer" containerID="74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff" Jul 07 13:08:07.037344 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.037323 2578 scope.go:117] "RemoveContainer" containerID="74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff" Jul 07 13:08:07.037611 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:08:07.037590 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff\": container with ID starting with 74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff not found: ID does not exist" containerID="74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff" Jul 07 13:08:07.037682 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.037618 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff"} err="failed to get container status \"74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff\": rpc error: code = NotFound desc = could not find container \"74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff\": container with ID starting with 74f3e5fb1c2daf9e9aba09990bafbaed59a7bc65df453126ffdb13758daba5ff not found: ID does not exist" Jul 07 13:08:07.052201 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.052176 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj"] Jul 07 13:08:07.056768 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.056744 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-087366f3-openshift-default-7dd5f9cbbc-nsplj"] Jul 07 13:08:07.076466 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.076442 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:07.076466 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.076469 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:07.076651 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.076479 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:07.076651 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.076487 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:07.076651 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.076498 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqdq6\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-kube-api-access-sqdq6\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:07.076651 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.076508 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/151c5ffc-94ee-48bd-9853-a142ac8b8f79-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:08:07.157057 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:08:07.157020 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="151c5ffc-94ee-48bd-9853-a142ac8b8f79" path="/var/lib/kubelet/pods/151c5ffc-94ee-48bd-9853-a142ac8b8f79/volumes" Jul 07 13:09:45.541234 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.541199 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx"] Jul 07 13:09:45.541670 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.541554 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="151c5ffc-94ee-48bd-9853-a142ac8b8f79" containerName="istio-proxy" Jul 07 13:09:45.541670 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.541566 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="151c5ffc-94ee-48bd-9853-a142ac8b8f79" containerName="istio-proxy" Jul 07 13:09:45.541670 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.541631 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="151c5ffc-94ee-48bd-9853-a142ac8b8f79" containerName="istio-proxy" Jul 07 13:09:45.543431 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.543415 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.547278 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.547258 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3c66f600-openshift-default-dockercfg-9vgrn\"" Jul 07 13:09:45.577548 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.577518 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx"] Jul 07 13:09:45.648426 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v2wt\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-kube-api-access-9v2wt\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648627 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648491 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-envoy\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648627 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istiod-ca-cert\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648627 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-data\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648751 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-socket\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648751 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648671 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-credential-socket\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648751 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-certs\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648751 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-token\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.648899 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.648782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-podinfo\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.749858 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.749814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-credential-socket\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.749858 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.749860 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-certs\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750245 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.749891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-token\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750245 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.749925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-podinfo\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750245 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.749956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v2wt\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-kube-api-access-9v2wt\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750245 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-envoy\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750245 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istiod-ca-cert\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750245 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750113 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-data\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750245 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-socket\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750590 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750369 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-credential-socket\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750590 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750551 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-data\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750590 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-socket\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750726 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750641 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-certs\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.750769 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.750722 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istiod-ca-cert\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.752458 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.752432 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-envoy\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.752624 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.752606 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-podinfo\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.759633 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.759611 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v2wt\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-kube-api-access-9v2wt\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.763359 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.763333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-token\") pod \"e2e-mt-3c66f600-openshift-default-585c88f79-gjktx\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.853250 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.853211 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:45.995471 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:45.995441 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx"] Jul 07 13:09:45.998095 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:09:45.998068 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f699d8b_1474_4767_8d77_4d2acbd6f78e.slice/crio-27a4644ce125bbb8c72a7f18eedf3217aacb7c4295fda06ce7e58c7489e2f846 WatchSource:0}: Error finding container 27a4644ce125bbb8c72a7f18eedf3217aacb7c4295fda06ce7e58c7489e2f846: Status 404 returned error can't find the container with id 27a4644ce125bbb8c72a7f18eedf3217aacb7c4295fda06ce7e58c7489e2f846 Jul 07 13:09:46.000222 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:46.000188 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:09:46.000305 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:46.000275 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:09:46.000346 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:46.000308 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:09:46.470476 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:46.470444 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" event={"ID":"3f699d8b-1474-4767-8d77-4d2acbd6f78e","Type":"ContainerStarted","Data":"7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1"} Jul 07 13:09:46.470653 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:46.470497 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" event={"ID":"3f699d8b-1474-4767-8d77-4d2acbd6f78e","Type":"ContainerStarted","Data":"27a4644ce125bbb8c72a7f18eedf3217aacb7c4295fda06ce7e58c7489e2f846"} Jul 07 13:09:46.500840 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:46.500779 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" podStartSLOduration=1.500760524 podStartE2EDuration="1.500760524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:09:46.499777839 +0000 UTC m=+1659.844432023" watchObservedRunningTime="2026-07-07 13:09:46.500760524 +0000 UTC m=+1659.845414709" Jul 07 13:09:46.854140 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:46.854098 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:47.858869 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:47.858841 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:48.479986 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:48.479950 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:09:48.480955 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:09:48.480937 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:10:02.545633 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:02.545539 2578 generic.go:356] "Generic (PLEG): container finished" podID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerID="7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1" exitCode=0 Jul 07 13:10:02.545633 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:02.545585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" event={"ID":"3f699d8b-1474-4767-8d77-4d2acbd6f78e","Type":"ContainerDied","Data":"7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1"} Jul 07 13:10:02.546103 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:02.546001 2578 scope.go:117] "RemoveContainer" containerID="7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1" Jul 07 13:10:02.546997 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:02.546971 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:02.547077 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:02.547063 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:02.547131 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:02.547101 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:03.480325 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:03.480288 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:10:03.552842 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:03.552799 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" event={"ID":"3f699d8b-1474-4767-8d77-4d2acbd6f78e","Type":"ContainerStarted","Data":"4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2"} Jul 07 13:10:03.854437 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:03.854396 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:10:03.855897 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:03.855847 2578 patch_prober.go:28] interesting pod/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 07 13:10:03.856064 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:03.855957 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 07 13:10:04.854461 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:04.854410 2578 patch_prober.go:28] interesting pod/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 07 13:10:04.855088 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:04.854500 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 07 13:10:05.854385 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:05.854345 2578 patch_prober.go:28] interesting pod/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 07 13:10:05.854801 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:05.854770 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 07 13:10:06.854161 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:06.854117 2578 patch_prober.go:28] interesting pod/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 07 13:10:06.854516 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:06.854481 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 07 13:10:07.854178 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:07.854126 2578 patch_prober.go:28] interesting pod/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 07 13:10:07.854706 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:07.854203 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 07 13:10:08.082828 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:08.082794 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx"] Jul 07 13:10:08.083073 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:08.083029 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" containerID="cri-o://4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2" gracePeriod=30 Jul 07 13:10:13.346712 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.346649 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:10:13.421793 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421753 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-credential-socket\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.421995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421843 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v2wt\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-kube-api-access-9v2wt\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.421995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421865 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-socket\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.421995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421880 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-envoy\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.421995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421896 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-data\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.421995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421920 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-token\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.421995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421958 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-certs\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.421995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.421992 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istiod-ca-cert\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.422372 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.422024 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-podinfo\") pod \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\" (UID: \"3f699d8b-1474-4767-8d77-4d2acbd6f78e\") " Jul 07 13:10:13.422372 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.422121 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:10:13.422492 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.422375 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.422492 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.422370 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:10:13.422492 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.422463 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:10:13.422636 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.422528 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:10:13.422939 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.422914 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:10:13.424281 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.424254 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:10:13.424281 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.424265 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:10:13.424403 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.424326 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-kube-api-access-9v2wt" (OuterVolumeSpecName: "kube-api-access-9v2wt") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "kube-api-access-9v2wt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:10:13.424438 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.424430 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f699d8b-1474-4767-8d77-4d2acbd6f78e" (UID: "3f699d8b-1474-4767-8d77-4d2acbd6f78e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:10:13.523434 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523393 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v2wt\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-kube-api-access-9v2wt\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.523434 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523428 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.523434 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523438 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.523434 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523447 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.523709 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523456 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.523709 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523463 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f699d8b-1474-4767-8d77-4d2acbd6f78e-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.523709 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523471 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.523709 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.523478 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f699d8b-1474-4767-8d77-4d2acbd6f78e-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:10:13.602905 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.602818 2578 generic.go:356] "Generic (PLEG): container finished" podID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerID="4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2" exitCode=0 Jul 07 13:10:13.602905 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.602878 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" Jul 07 13:10:13.602905 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.602885 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" event={"ID":"3f699d8b-1474-4767-8d77-4d2acbd6f78e","Type":"ContainerDied","Data":"4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2"} Jul 07 13:10:13.603202 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.602952 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx" event={"ID":"3f699d8b-1474-4767-8d77-4d2acbd6f78e","Type":"ContainerDied","Data":"27a4644ce125bbb8c72a7f18eedf3217aacb7c4295fda06ce7e58c7489e2f846"} Jul 07 13:10:13.603202 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.602974 2578 scope.go:117] "RemoveContainer" containerID="4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2" Jul 07 13:10:13.612111 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.612096 2578 scope.go:117] "RemoveContainer" containerID="7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1" Jul 07 13:10:13.620270 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.620245 2578 scope.go:117] "RemoveContainer" containerID="4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2" Jul 07 13:10:13.620569 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:10:13.620549 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2\": container with ID starting with 4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2 not found: ID does not exist" containerID="4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2" Jul 07 13:10:13.620650 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.620578 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2"} err="failed to get container status \"4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2\": rpc error: code = NotFound desc = could not find container \"4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2\": container with ID starting with 4ce95daa8fe15bf2187eef34bec85eed206ccc46da8a102a3ed0cdd72de795e2 not found: ID does not exist" Jul 07 13:10:13.620650 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.620597 2578 scope.go:117] "RemoveContainer" containerID="7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1" Jul 07 13:10:13.620873 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:10:13.620848 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1\": container with ID starting with 7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1 not found: ID does not exist" containerID="7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1" Jul 07 13:10:13.620920 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.620885 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1"} err="failed to get container status \"7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1\": rpc error: code = NotFound desc = could not find container \"7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1\": container with ID starting with 7eb916b4bd03ebc286072e58137a345569c2d15c9c4afe199b83f5a5b5168aa1 not found: ID does not exist" Jul 07 13:10:13.627032 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.627007 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx"] Jul 07 13:10:13.633391 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:13.633366 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3c66f600-openshift-default-585c88f79-gjktx"] Jul 07 13:10:15.157817 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:15.157779 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" path="/var/lib/kubelet/pods/3f699d8b-1474-4767-8d77-4d2acbd6f78e/volumes" Jul 07 13:10:28.886892 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:28.886858 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh"] Jul 07 13:10:28.887301 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:28.887253 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" Jul 07 13:10:28.887301 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:28.887270 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" Jul 07 13:10:28.887383 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:28.887356 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" Jul 07 13:10:28.890425 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:28.890407 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:28.893003 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:28.892983 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7dc7bf8d-openshift-default-dockercfg-m64qr\"" Jul 07 13:10:28.901573 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:28.901549 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh"] Jul 07 13:10:29.054385 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-istio-token\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054409 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-envoy\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-data\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7ab536-301f-4a50-b673-e706909d3b58-istiod-ca-cert\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-credential-socket\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79985\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-kube-api-access-79985\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-socket\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-certs\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.054842 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.054636 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7ab536-301f-4a50-b673-e706909d3b58-istio-podinfo\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155118 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155010 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-credential-socket\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155118 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79985\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-kube-api-access-79985\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155118 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-socket\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155118 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155097 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-certs\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155445 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7ab536-301f-4a50-b673-e706909d3b58-istio-podinfo\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155445 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-istio-token\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155445 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155267 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-envoy\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155445 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-data\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155445 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7ab536-301f-4a50-b673-e706909d3b58-istiod-ca-cert\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155684 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155519 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-credential-socket\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155684 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155536 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-socket\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155684 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155665 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-certs\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.155906 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.155817 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-data\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.156077 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.156057 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7ab536-301f-4a50-b673-e706909d3b58-istiod-ca-cert\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.157767 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.157739 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-envoy\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.157967 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.157950 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7ab536-301f-4a50-b673-e706909d3b58-istio-podinfo\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.163534 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.163511 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-istio-token\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.163703 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.163677 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79985\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-kube-api-access-79985\") pod \"e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.201898 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.201864 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:29.338685 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.338660 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh"] Jul 07 13:10:29.341206 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:10:29.341178 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7ab536_301f_4a50_b673_e706909d3b58.slice/crio-c681604c0d2dfdd2e29845cf02ea56ec82e1e3bd302f8b7b5ad783cfd8cc256c WatchSource:0}: Error finding container c681604c0d2dfdd2e29845cf02ea56ec82e1e3bd302f8b7b5ad783cfd8cc256c: Status 404 returned error can't find the container with id c681604c0d2dfdd2e29845cf02ea56ec82e1e3bd302f8b7b5ad783cfd8cc256c Jul 07 13:10:29.343269 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.343235 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:29.343366 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.343320 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:29.343428 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.343375 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:29.682372 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.682336 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" event={"ID":"cb7ab536-301f-4a50-b673-e706909d3b58","Type":"ContainerStarted","Data":"beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4"} Jul 07 13:10:29.682591 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.682392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" event={"ID":"cb7ab536-301f-4a50-b673-e706909d3b58","Type":"ContainerStarted","Data":"c681604c0d2dfdd2e29845cf02ea56ec82e1e3bd302f8b7b5ad783cfd8cc256c"} Jul 07 13:10:29.711560 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:29.711500 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" podStartSLOduration=1.711481314 podStartE2EDuration="1.711481314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:10:29.710974165 +0000 UTC m=+1703.055628351" watchObservedRunningTime="2026-07-07 13:10:29.711481314 +0000 UTC m=+1703.056135499" Jul 07 13:10:30.202218 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:30.202178 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:30.207176 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:30.207151 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:30.687071 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:30.687025 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:30.687936 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:30.687918 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:45.689497 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.689466 2578 patch_prober.go:28] interesting pod/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 13:10:45.689909 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.689545 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 13:10:45.764373 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.764336 2578 generic.go:356] "Generic (PLEG): container finished" podID="cb7ab536-301f-4a50-b673-e706909d3b58" containerID="beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4" exitCode=0 Jul 07 13:10:45.764549 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.764408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" event={"ID":"cb7ab536-301f-4a50-b673-e706909d3b58","Type":"ContainerDied","Data":"beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4"} Jul 07 13:10:45.764837 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.764824 2578 scope.go:117] "RemoveContainer" containerID="beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4" Jul 07 13:10:45.765828 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.765800 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:45.765891 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.765869 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:45.765927 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:45.765908 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:10:46.771164 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:46.771128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" event={"ID":"cb7ab536-301f-4a50-b673-e706909d3b58","Type":"ContainerStarted","Data":"65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc"} Jul 07 13:10:47.202731 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:47.202694 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:47.202918 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:47.202890 2578 patch_prober.go:28] interesting pod/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 13:10:47.202968 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:47.202937 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 13:10:48.202579 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:48.202535 2578 patch_prober.go:28] interesting pod/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 13:10:48.203104 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:48.202602 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 13:10:49.202788 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:49.202751 2578 patch_prober.go:28] interesting pod/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 13:10:49.203227 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:49.202804 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 13:10:50.203180 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:50.203141 2578 patch_prober.go:28] interesting pod/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body= Jul 07 13:10:50.203666 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:50.203242 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" Jul 07 13:10:51.206429 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:51.206399 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:51.206807 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:51.206692 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:10:51.207339 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:10:51.207318 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:12:00.665279 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:00.665196 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh"] Jul 07 13:12:00.665758 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:00.665483 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" containerID="cri-o://65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc" gracePeriod=30 Jul 07 13:12:05.924737 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:05.924675 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:12:06.042674 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042634 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7ab536-301f-4a50-b673-e706909d3b58-istiod-ca-cert\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.042674 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042673 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-socket\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.042932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042694 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7ab536-301f-4a50-b673-e706909d3b58-istio-podinfo\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.042932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042719 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-envoy\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.042932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042740 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-data\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.042932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042882 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-credential-socket\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.043165 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042941 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-istio-token\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.043165 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.042988 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79985\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-kube-api-access-79985\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.043165 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.043072 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:12:06.043165 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.043149 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-certs\") pod \"cb7ab536-301f-4a50-b673-e706909d3b58\" (UID: \"cb7ab536-301f-4a50-b673-e706909d3b58\") " Jul 07 13:12:06.043503 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.043460 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.043503 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.043147 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb7ab536-301f-4a50-b673-e706909d3b58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:12:06.043503 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.043392 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:12:06.043723 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.043589 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:12:06.043841 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.043810 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:12:06.045208 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.045183 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:12:06.045319 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.045232 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb7ab536-301f-4a50-b673-e706909d3b58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:12:06.045498 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.045478 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:12:06.045576 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.045539 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-kube-api-access-79985" (OuterVolumeSpecName: "kube-api-access-79985") pod "cb7ab536-301f-4a50-b673-e706909d3b58" (UID: "cb7ab536-301f-4a50-b673-e706909d3b58"). InnerVolumeSpecName "kube-api-access-79985". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:12:06.132627 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.132590 2578 generic.go:356] "Generic (PLEG): container finished" podID="cb7ab536-301f-4a50-b673-e706909d3b58" containerID="65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc" exitCode=0 Jul 07 13:12:06.132854 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.132641 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" Jul 07 13:12:06.132854 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.132664 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" event={"ID":"cb7ab536-301f-4a50-b673-e706909d3b58","Type":"ContainerDied","Data":"65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc"} Jul 07 13:12:06.132854 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.132715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh" event={"ID":"cb7ab536-301f-4a50-b673-e706909d3b58","Type":"ContainerDied","Data":"c681604c0d2dfdd2e29845cf02ea56ec82e1e3bd302f8b7b5ad783cfd8cc256c"} Jul 07 13:12:06.132854 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.132740 2578 scope.go:117] "RemoveContainer" containerID="65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc" Jul 07 13:12:06.141995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.141976 2578 scope.go:117] "RemoveContainer" containerID="beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4" Jul 07 13:12:06.144551 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144527 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7ab536-301f-4a50-b673-e706909d3b58-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.144638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144557 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7ab536-301f-4a50-b673-e706909d3b58-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.144638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144571 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.144638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144583 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.144638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144596 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.144638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144609 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.144638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144621 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79985\" (UniqueName: \"kubernetes.io/projected/cb7ab536-301f-4a50-b673-e706909d3b58-kube-api-access-79985\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.144638 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.144634 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7ab536-301f-4a50-b673-e706909d3b58-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:12:06.150352 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.150332 2578 scope.go:117] "RemoveContainer" containerID="65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc" Jul 07 13:12:06.150632 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:12:06.150612 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc\": container with ID starting with 65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc not found: ID does not exist" containerID="65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc" Jul 07 13:12:06.150676 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.150641 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc"} err="failed to get container status \"65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc\": rpc error: code = NotFound desc = could not find container \"65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc\": container with ID starting with 65ab0f4d4a01e222ac5158f1555dfc7b245c3942490bb08c3c09320055204dcc not found: ID does not exist" Jul 07 13:12:06.150676 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.150658 2578 scope.go:117] "RemoveContainer" containerID="beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4" Jul 07 13:12:06.150893 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:12:06.150874 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4\": container with ID starting with beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4 not found: ID does not exist" containerID="beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4" Jul 07 13:12:06.150932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.150900 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4"} err="failed to get container status \"beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4\": rpc error: code = NotFound desc = could not find container \"beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4\": container with ID starting with beb68b1af7a9127adc07c1556837a6b69e992bf13b6a8dccb397bd67d7ac3ed4 not found: ID does not exist" Jul 07 13:12:06.156286 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.156264 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh"] Jul 07 13:12:06.157787 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:06.157768 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7dc7bf8d-openshift-default-6cdc7f67c5-7fndh"] Jul 07 13:12:07.162815 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:07.162777 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" path="/var/lib/kubelet/pods/cb7ab536-301f-4a50-b673-e706909d3b58/volumes" Jul 07 13:12:07.163265 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:07.162919 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:12:07.166812 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:07.166785 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:12:57.926590 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926535 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7"] Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926881 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926905 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926918 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926923 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926932 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926937 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926991 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.926997 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f699d8b-1474-4767-8d77-4d2acbd6f78e" containerName="istio-proxy" Jul 07 13:12:57.927078 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.927004 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb7ab536-301f-4a50-b673-e706909d3b58" containerName="istio-proxy" Jul 07 13:12:57.929940 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.929922 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:57.933063 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.933031 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6b8022-openshift-default-dockercfg-jvmmp\"" Jul 07 13:12:57.951011 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:57.950971 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7"] Jul 07 13:12:58.102552 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-socket\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102552 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-envoy\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102622 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-certs\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-token\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-data\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102759 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-credential-socket\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102906 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-podinfo\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102906 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istiod-ca-cert\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.102906 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.102831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdjw7\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-kube-api-access-kdjw7\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204235 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-data\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204235 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-credential-socket\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204235 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-podinfo\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204235 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204233 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istiod-ca-cert\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdjw7\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-kube-api-access-kdjw7\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-socket\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-envoy\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-certs\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204566 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-token\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204817 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204656 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-credential-socket\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204817 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204685 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-data\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204817 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204769 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-socket\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.204983 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.204955 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-certs\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.205098 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.205079 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istiod-ca-cert\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.206727 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.206709 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-envoy\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.206824 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.206710 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-podinfo\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.216810 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.216780 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdjw7\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-kube-api-access-kdjw7\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.217227 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.217208 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-token\") pod \"e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.240102 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.240072 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:12:58.384251 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.384220 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7"] Jul 07 13:12:58.386293 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:12:58.386261 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6f4ee8_4103_44a1_bbe8_35be63386a9b.slice/crio-8c21185b3a67676ac4960241ae5417dba2dfbcc3edb09ae4aa41d4dcbd4a6953 WatchSource:0}: Error finding container 8c21185b3a67676ac4960241ae5417dba2dfbcc3edb09ae4aa41d4dcbd4a6953: Status 404 returned error can't find the container with id 8c21185b3a67676ac4960241ae5417dba2dfbcc3edb09ae4aa41d4dcbd4a6953 Jul 07 13:12:58.388287 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.388257 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:12:58.388376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.388322 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:12:58.388376 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:58.388351 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:12:59.377235 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:59.377199 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" event={"ID":"7c6f4ee8-4103-44a1-bbe8-35be63386a9b","Type":"ContainerStarted","Data":"8c9c72cf6b52822ea1cc88b701d7895ebb186f72bffaa4a9fcbb6de02c265879"} Jul 07 13:12:59.377729 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:59.377256 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" event={"ID":"7c6f4ee8-4103-44a1-bbe8-35be63386a9b","Type":"ContainerStarted","Data":"8c21185b3a67676ac4960241ae5417dba2dfbcc3edb09ae4aa41d4dcbd4a6953"} Jul 07 13:12:59.400696 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:12:59.400652 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" podStartSLOduration=2.400635343 podStartE2EDuration="2.400635343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:12:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:12:59.399409665 +0000 UTC m=+1852.744063862" watchObservedRunningTime="2026-07-07 13:12:59.400635343 +0000 UTC m=+1852.745289526" Jul 07 13:13:00.240568 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:00.240524 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:00.245465 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:00.245435 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:00.382127 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:00.382097 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:00.382969 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:00.382950 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:06.634441 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.634406 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j"] Jul 07 13:13:06.638115 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.638093 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.641018 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.640998 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e2f0d5-openshift-default-dockercfg-hppk8\"" Jul 07 13:13:06.656167 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.656135 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j"] Jul 07 13:13:06.782713 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.782677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-certs\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.782713 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.782720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-data\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.782920 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.782740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-token\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.782920 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.782867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a6d3c2-527b-4952-b427-d131c450cbe8-istiod-ca-cert\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.782920 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.782902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-socket\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.783019 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.782924 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-credential-socket\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.783019 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.782987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-podinfo\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.783105 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.783028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqcl2\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-kube-api-access-xqcl2\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.783105 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.783092 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-envoy\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884103 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a6d3c2-527b-4952-b427-d131c450cbe8-istiod-ca-cert\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884103 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-socket\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-credential-socket\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-podinfo\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqcl2\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-kube-api-access-xqcl2\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884211 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-envoy\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-certs\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884304 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-data\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-token\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884754 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884559 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-socket\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884754 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884734 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-certs\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884832 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884778 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-data\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884832 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884811 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-credential-socket\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.884899 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.884869 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a6d3c2-527b-4952-b427-d131c450cbe8-istiod-ca-cert\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.886579 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.886554 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-envoy\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.886840 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.886821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-podinfo\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.893471 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.893447 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-token\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.893661 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.893638 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqcl2\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-kube-api-access-xqcl2\") pod \"e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:06.948612 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:06.948574 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:07.089826 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.089802 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j"] Jul 07 13:13:07.091787 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:13:07.091760 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a6d3c2_527b_4952_b427_d131c450cbe8.slice/crio-b67af52a44f9c263f328039a33cdc792696e4fd98f88e9cf2fa9d73704c4819e WatchSource:0}: Error finding container b67af52a44f9c263f328039a33cdc792696e4fd98f88e9cf2fa9d73704c4819e: Status 404 returned error can't find the container with id b67af52a44f9c263f328039a33cdc792696e4fd98f88e9cf2fa9d73704c4819e Jul 07 13:13:07.093961 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.093929 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:07.094092 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.094000 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:07.094092 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.094065 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:07.418093 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.417982 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" event={"ID":"73a6d3c2-527b-4952-b427-d131c450cbe8","Type":"ContainerStarted","Data":"0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779"} Jul 07 13:13:07.418236 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.418097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" event={"ID":"73a6d3c2-527b-4952-b427-d131c450cbe8","Type":"ContainerStarted","Data":"b67af52a44f9c263f328039a33cdc792696e4fd98f88e9cf2fa9d73704c4819e"} Jul 07 13:13:07.442173 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.442119 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" podStartSLOduration=1.442100741 podStartE2EDuration="1.442100741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:13:07.441286603 +0000 UTC m=+1860.785940788" watchObservedRunningTime="2026-07-07 13:13:07.442100741 +0000 UTC m=+1860.786754924" Jul 07 13:13:07.949700 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:07.949663 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:08.954304 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:08.954271 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:09.427084 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:09.427028 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:09.428204 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:09.428180 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:16.423248 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:16.423205 2578 patch_prober.go:28] interesting pod/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 13:13:16.423775 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:16.423279 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 13:13:17.481090 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:17.481031 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerID="8c9c72cf6b52822ea1cc88b701d7895ebb186f72bffaa4a9fcbb6de02c265879" exitCode=0 Jul 07 13:13:17.481522 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:17.481070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" event={"ID":"7c6f4ee8-4103-44a1-bbe8-35be63386a9b","Type":"ContainerDied","Data":"8c9c72cf6b52822ea1cc88b701d7895ebb186f72bffaa4a9fcbb6de02c265879"} Jul 07 13:13:17.481619 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:17.481603 2578 scope.go:117] "RemoveContainer" containerID="8c9c72cf6b52822ea1cc88b701d7895ebb186f72bffaa4a9fcbb6de02c265879" Jul 07 13:13:17.482691 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:17.482660 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:17.482766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:17.482750 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:17.482808 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:17.482790 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:18.489283 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:18.489245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" event={"ID":"7c6f4ee8-4103-44a1-bbe8-35be63386a9b","Type":"ContainerStarted","Data":"cde3ccf56914d34474af996c08600b8e47370eb9758a2501e1143fba759cca88"} Jul 07 13:13:19.240550 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:19.240508 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:19.240812 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:19.240781 2578 patch_prober.go:28] interesting pod/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 13:13:19.240899 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:19.240845 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 13:13:20.241577 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:20.241531 2578 patch_prober.go:28] interesting pod/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 13:13:20.242034 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:20.241602 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 13:13:21.243088 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:21.243005 2578 patch_prober.go:28] interesting pod/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 13:13:21.243575 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:21.243094 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 13:13:22.241500 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:22.241457 2578 patch_prober.go:28] interesting pod/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 13:13:22.241731 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:22.241530 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 13:13:23.244392 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:23.244365 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:23.244789 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:23.244701 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:23.245662 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:23.245640 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:13:24.428411 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.428373 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 13:13:24.428835 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.428429 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 13:13:24.517886 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.517854 2578 generic.go:356] "Generic (PLEG): container finished" podID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerID="0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779" exitCode=0 Jul 07 13:13:24.518088 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.517946 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" event={"ID":"73a6d3c2-527b-4952-b427-d131c450cbe8","Type":"ContainerDied","Data":"0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779"} Jul 07 13:13:24.518477 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.518467 2578 scope.go:117] "RemoveContainer" containerID="0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779" Jul 07 13:13:24.519439 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.519414 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:24.519495 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.519477 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:24.519544 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:24.519509 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:13:25.529303 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:25.529267 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" event={"ID":"73a6d3c2-527b-4952-b427-d131c450cbe8","Type":"ContainerStarted","Data":"9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165"} Jul 07 13:13:25.948869 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:25.948832 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:25.949220 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:25.949187 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 13:13:25.949352 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:25.949267 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 13:13:26.949291 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:26.949244 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 13:13:26.949837 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:26.949328 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 13:13:27.949254 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:27.949212 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 13:13:27.949593 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:27.949559 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 13:13:28.950128 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:28.950090 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 07 13:13:28.950835 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:28.950803 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 07 13:13:29.954125 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:29.954091 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:29.954552 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:29.954365 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:13:29.955111 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:13:29.955096 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:15:01.804488 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:01.804408 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7"] Jul 07 13:15:01.804944 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:01.804641 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" containerID="cri-o://cde3ccf56914d34474af996c08600b8e47370eb9758a2501e1143fba759cca88" gracePeriod=30 Jul 07 13:15:07.028262 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.028212 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerID="cde3ccf56914d34474af996c08600b8e47370eb9758a2501e1143fba759cca88" exitCode=0 Jul 07 13:15:07.028761 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.028274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" event={"ID":"7c6f4ee8-4103-44a1-bbe8-35be63386a9b","Type":"ContainerDied","Data":"cde3ccf56914d34474af996c08600b8e47370eb9758a2501e1143fba759cca88"} Jul 07 13:15:07.028761 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.028338 2578 scope.go:117] "RemoveContainer" containerID="8c9c72cf6b52822ea1cc88b701d7895ebb186f72bffaa4a9fcbb6de02c265879" Jul 07 13:15:07.063595 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.063567 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:15:07.238380 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238337 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-podinfo\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.238585 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238393 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-certs\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.238585 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238412 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-credential-socket\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.238585 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238432 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-token\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.238585 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238449 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdjw7\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-kube-api-access-kdjw7\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.238585 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238487 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-socket\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.238585 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238520 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-data\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.238585 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238543 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istiod-ca-cert\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.239193 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238627 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-envoy\") pod \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\" (UID: \"7c6f4ee8-4103-44a1-bbe8-35be63386a9b\") " Jul 07 13:15:07.239193 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238823 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:07.239193 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238877 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:07.239193 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.238925 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:07.239193 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.239124 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:07.239571 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.239547 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:07.240949 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.240929 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:07.241110 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.240985 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:07.241264 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.241241 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-kube-api-access-kdjw7" (OuterVolumeSpecName: "kube-api-access-kdjw7") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "kube-api-access-kdjw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:07.241348 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.241255 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c6f4ee8-4103-44a1-bbe8-35be63386a9b" (UID: "7c6f4ee8-4103-44a1-bbe8-35be63386a9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:15:07.340475 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340430 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340475 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340468 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340475 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340482 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340498 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340512 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdjw7\" (UniqueName: \"kubernetes.io/projected/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-kube-api-access-kdjw7\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340527 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340541 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340555 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:07.340765 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:07.340569 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6f4ee8-4103-44a1-bbe8-35be63386a9b-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:08.034949 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:08.034910 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" event={"ID":"7c6f4ee8-4103-44a1-bbe8-35be63386a9b","Type":"ContainerDied","Data":"8c21185b3a67676ac4960241ae5417dba2dfbcc3edb09ae4aa41d4dcbd4a6953"} Jul 07 13:15:08.034949 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:08.034935 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7" Jul 07 13:15:08.035498 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:08.034979 2578 scope.go:117] "RemoveContainer" containerID="cde3ccf56914d34474af996c08600b8e47370eb9758a2501e1143fba759cca88" Jul 07 13:15:08.070961 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:08.070924 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7"] Jul 07 13:15:08.076322 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:08.076288 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8022-openshift-default-6b4f47979c-tzct7"] Jul 07 13:15:08.801684 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:08.801644 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j"] Jul 07 13:15:08.801948 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:08.801920 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" containerID="cri-o://9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165" gracePeriod=30 Jul 07 13:15:09.156865 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.156821 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" path="/var/lib/kubelet/pods/7c6f4ee8-4103-44a1-bbe8-35be63386a9b/volumes" Jul 07 13:15:09.789084 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.789028 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp"] Jul 07 13:15:09.789459 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.789443 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" Jul 07 13:15:09.789570 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.789463 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" Jul 07 13:15:09.789623 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.789586 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" Jul 07 13:15:09.789623 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.789600 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" Jul 07 13:15:09.792932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.792904 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.796508 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.796484 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8ff4d1-openshift-default-dockercfg-v7s4k\"" Jul 07 13:15:09.812586 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.812556 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp"] Jul 07 13:15:09.860153 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-socket\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860348 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzmwt\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-kube-api-access-qzmwt\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860348 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-credential-socket\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860348 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-certs\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860348 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-data\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860499 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-envoy\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860499 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15598e31-9727-42e5-a48a-168e2e174b35-istiod-ca-cert\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860499 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15598e31-9727-42e5-a48a-168e2e174b35-istio-podinfo\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.860499 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.860489 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-istio-token\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.960891 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.960857 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-envoy\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.960891 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.960894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15598e31-9727-42e5-a48a-168e2e174b35-istiod-ca-cert\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.960915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15598e31-9727-42e5-a48a-168e2e174b35-istio-podinfo\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.960930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-istio-token\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.960957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-socket\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.960986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzmwt\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-kube-api-access-qzmwt\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-credential-socket\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-certs\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-data\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961561 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961397 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-socket\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961561 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961444 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-credential-socket\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961561 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961515 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-data\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961720 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961590 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-certs\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.961915 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.961891 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15598e31-9727-42e5-a48a-168e2e174b35-istiod-ca-cert\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.963518 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.963494 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-envoy\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.963619 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.963599 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15598e31-9727-42e5-a48a-168e2e174b35-istio-podinfo\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.972208 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.972184 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-istio-token\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:09.972702 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:09.972681 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzmwt\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-kube-api-access-qzmwt\") pod \"e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:10.102976 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:10.102937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:10.239819 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:10.239775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp"] Jul 07 13:15:10.241953 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:15:10.241928 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15598e31_9727_42e5_a48a_168e2e174b35.slice/crio-179af93642dcd231f6f3a0aadfb4677700151b75e0995808c54d2cacb3ea87f5 WatchSource:0}: Error finding container 179af93642dcd231f6f3a0aadfb4677700151b75e0995808c54d2cacb3ea87f5: Status 404 returned error can't find the container with id 179af93642dcd231f6f3a0aadfb4677700151b75e0995808c54d2cacb3ea87f5 Jul 07 13:15:10.244163 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:10.244134 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:10.244234 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:10.244200 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:10.244274 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:10.244239 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:11.053074 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:11.053020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" event={"ID":"15598e31-9727-42e5-a48a-168e2e174b35","Type":"ContainerStarted","Data":"f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb"} Jul 07 13:15:11.053289 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:11.053102 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" event={"ID":"15598e31-9727-42e5-a48a-168e2e174b35","Type":"ContainerStarted","Data":"179af93642dcd231f6f3a0aadfb4677700151b75e0995808c54d2cacb3ea87f5"} Jul 07 13:15:11.077380 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:11.077332 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" podStartSLOduration=2.077313306 podStartE2EDuration="2.077313306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:11.075017228 +0000 UTC m=+1984.419671413" watchObservedRunningTime="2026-07-07 13:15:11.077313306 +0000 UTC m=+1984.421967489" Jul 07 13:15:11.103106 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:11.103064 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:11.107932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:11.107906 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:12.058118 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:12.058087 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:12.059122 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:12.059105 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:14.056743 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.056713 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:15:14.068920 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.068888 2578 generic.go:356] "Generic (PLEG): container finished" podID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerID="9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165" exitCode=0 Jul 07 13:15:14.069061 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.068939 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" Jul 07 13:15:14.069061 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.068966 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" event={"ID":"73a6d3c2-527b-4952-b427-d131c450cbe8","Type":"ContainerDied","Data":"9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165"} Jul 07 13:15:14.069061 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.069020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j" event={"ID":"73a6d3c2-527b-4952-b427-d131c450cbe8","Type":"ContainerDied","Data":"b67af52a44f9c263f328039a33cdc792696e4fd98f88e9cf2fa9d73704c4819e"} Jul 07 13:15:14.069191 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.069060 2578 scope.go:117] "RemoveContainer" containerID="9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165" Jul 07 13:15:14.078746 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.078725 2578 scope.go:117] "RemoveContainer" containerID="0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779" Jul 07 13:15:14.088850 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.088828 2578 scope.go:117] "RemoveContainer" containerID="9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165" Jul 07 13:15:14.089200 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:15:14.089177 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165\": container with ID starting with 9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165 not found: ID does not exist" containerID="9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165" Jul 07 13:15:14.089267 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.089209 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165"} err="failed to get container status \"9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165\": rpc error: code = NotFound desc = could not find container \"9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165\": container with ID starting with 9deb3eee958cf023fa66b112a158630624f0d6da3b0b53e43ec5012f66086165 not found: ID does not exist" Jul 07 13:15:14.089267 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.089229 2578 scope.go:117] "RemoveContainer" containerID="0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779" Jul 07 13:15:14.089491 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:15:14.089468 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779\": container with ID starting with 0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779 not found: ID does not exist" containerID="0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779" Jul 07 13:15:14.089552 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.089501 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779"} err="failed to get container status \"0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779\": rpc error: code = NotFound desc = could not find container \"0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779\": container with ID starting with 0e5bd218407beb442525904c27b7897ccd415c36e06c4356beb05a368ae81779 not found: ID does not exist" Jul 07 13:15:14.098522 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098496 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-data\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.098759 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-socket\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.098851 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098841 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-token\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.098919 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098865 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a6d3c2-527b-4952-b427-d131c450cbe8-istiod-ca-cert\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.098919 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098890 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqcl2\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-kube-api-access-xqcl2\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.099033 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098923 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-envoy\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.099033 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098967 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-credential-socket\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.099033 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098994 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-podinfo\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.099283 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099070 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-certs\") pod \"73a6d3c2-527b-4952-b427-d131c450cbe8\" (UID: \"73a6d3c2-527b-4952-b427-d131c450cbe8\") " Jul 07 13:15:14.099283 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.098962 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:14.099435 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099337 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-data" (OuterVolumeSpecName: "istio-data") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:14.099435 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099356 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:14.099537 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099438 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.099537 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099457 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.099537 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099471 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.099837 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099815 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:14.100007 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.099982 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a6d3c2-527b-4952-b427-d131c450cbe8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:14.101504 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.101477 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-kube-api-access-xqcl2" (OuterVolumeSpecName: "kube-api-access-xqcl2") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "kube-api-access-xqcl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:14.101504 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.101488 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:15:14.101718 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.101697 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:14.101809 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.101791 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-token" (OuterVolumeSpecName: "istio-token") pod "73a6d3c2-527b-4952-b427-d131c450cbe8" (UID: "73a6d3c2-527b-4952-b427-d131c450cbe8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:14.200832 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.200796 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.200832 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.200824 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a6d3c2-527b-4952-b427-d131c450cbe8-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.200832 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.200840 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqcl2\" (UniqueName: \"kubernetes.io/projected/73a6d3c2-527b-4952-b427-d131c450cbe8-kube-api-access-xqcl2\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.201210 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.200855 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.201210 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.200864 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a6d3c2-527b-4952-b427-d131c450cbe8-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.201210 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.200872 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a6d3c2-527b-4952-b427-d131c450cbe8-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:14.391683 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.391651 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j"] Jul 07 13:15:14.395247 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:14.395220 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e2f0d5-openshift-default-7f894995f-4tk2j"] Jul 07 13:15:15.157389 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:15.157353 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" path="/var/lib/kubelet/pods/73a6d3c2-527b-4952-b427-d131c450cbe8/volumes" Jul 07 13:15:18.288024 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.287988 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb"] Jul 07 13:15:18.288529 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.288469 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" Jul 07 13:15:18.288529 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.288485 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6f4ee8-4103-44a1-bbe8-35be63386a9b" containerName="istio-proxy" Jul 07 13:15:18.288529 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.288495 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" Jul 07 13:15:18.288529 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.288501 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" Jul 07 13:15:18.288746 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.288566 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" Jul 07 13:15:18.288746 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.288575 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" Jul 07 13:15:18.291379 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.291359 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.294160 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.294139 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8e00d7-openshift-default-dockercfg-dxzk2\"" Jul 07 13:15:18.302760 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.302728 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb"] Jul 07 13:15:18.337343 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-data\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.337343 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z47s\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-kube-api-access-6z47s\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.337597 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-token\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.337597 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337427 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-certs\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.337597 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-podinfo\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.337597 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337520 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istiod-ca-cert\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.337597 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337581 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-socket\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.338085 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-credential-socket\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.338085 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.337644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-envoy\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439219 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-podinfo\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439219 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istiod-ca-cert\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439481 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439275 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-socket\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439481 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-credential-socket\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439481 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-envoy\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439481 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-data\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439481 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z47s\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-kube-api-access-6z47s\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439481 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-token\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439757 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-certs\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439757 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-credential-socket\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439858 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439786 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-socket\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439858 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439838 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-data\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.439973 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.439897 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-certs\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.440255 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.440219 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istiod-ca-cert\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.441661 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.441637 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-envoy\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.441902 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.441881 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-podinfo\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.448476 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.448451 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-token\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.448599 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.448498 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z47s\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-kube-api-access-6z47s\") pod \"e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.602882 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.602842 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:18.740632 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.740599 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb"] Jul 07 13:15:18.743286 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:15:18.743256 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2f1d1d_767b_4382_a94d_185d9dd269f2.slice/crio-545e8b8cc791ec506c31e83bdf3f75574edb62164fde6494072d8882913488ad WatchSource:0}: Error finding container 545e8b8cc791ec506c31e83bdf3f75574edb62164fde6494072d8882913488ad: Status 404 returned error can't find the container with id 545e8b8cc791ec506c31e83bdf3f75574edb62164fde6494072d8882913488ad Jul 07 13:15:18.745253 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.745222 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:18.745359 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.745285 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:18.745359 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:18.745313 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:19.097404 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:19.097370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" event={"ID":"1f2f1d1d-767b-4382-a94d-185d9dd269f2","Type":"ContainerStarted","Data":"2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585"} Jul 07 13:15:19.097590 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:19.097424 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" event={"ID":"1f2f1d1d-767b-4382-a94d-185d9dd269f2","Type":"ContainerStarted","Data":"545e8b8cc791ec506c31e83bdf3f75574edb62164fde6494072d8882913488ad"} Jul 07 13:15:19.123972 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:19.123922 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" podStartSLOduration=1.123903225 podStartE2EDuration="1.123903225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:19.122189536 +0000 UTC m=+1992.466843720" watchObservedRunningTime="2026-07-07 13:15:19.123903225 +0000 UTC m=+1992.468557408" Jul 07 13:15:19.603523 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:19.603485 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:19.608255 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:19.608230 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:20.102812 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:20.102782 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:20.104011 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:20.103990 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:28.100254 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:28.100201 2578 patch_prober.go:28] interesting pod/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 13:15:28.100663 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:28.100273 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 13:15:29.149156 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:29.149121 2578 generic.go:356] "Generic (PLEG): container finished" podID="15598e31-9727-42e5-a48a-168e2e174b35" containerID="f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb" exitCode=0 Jul 07 13:15:29.149156 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:29.149164 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" event={"ID":"15598e31-9727-42e5-a48a-168e2e174b35","Type":"ContainerDied","Data":"f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb"} Jul 07 13:15:29.149769 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:29.149683 2578 scope.go:117] "RemoveContainer" containerID="f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb" Jul 07 13:15:29.150787 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:29.150757 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:29.150883 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:29.150839 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:29.150945 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:29.150888 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:30.158888 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:30.158841 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" event={"ID":"15598e31-9727-42e5-a48a-168e2e174b35","Type":"ContainerStarted","Data":"88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e"} Jul 07 13:15:31.103308 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:31.103266 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:31.103511 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:31.103476 2578 patch_prober.go:28] interesting pod/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 13:15:31.103561 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:31.103526 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 13:15:32.104056 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:32.104009 2578 patch_prober.go:28] interesting pod/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 13:15:32.104542 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:32.104108 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 13:15:33.103612 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:33.103581 2578 patch_prober.go:28] interesting pod/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 13:15:33.103882 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:33.103855 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 13:15:34.104130 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:34.104090 2578 patch_prober.go:28] interesting pod/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 13:15:34.104678 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:34.104159 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 13:15:35.107231 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:35.107200 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:35.107668 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:35.107481 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:35.108321 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:35.108306 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:36.144211 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:36.144171 2578 patch_prober.go:28] interesting pod/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 13:15:36.144624 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:36.144251 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 13:15:37.202830 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:37.202797 2578 generic.go:356] "Generic (PLEG): container finished" podID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerID="2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585" exitCode=0 Jul 07 13:15:37.203287 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:37.202872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" event={"ID":"1f2f1d1d-767b-4382-a94d-185d9dd269f2","Type":"ContainerDied","Data":"2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585"} Jul 07 13:15:37.203349 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:37.203334 2578 scope.go:117] "RemoveContainer" containerID="2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585" Jul 07 13:15:37.204418 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:37.204393 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:37.204498 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:37.204459 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:37.204498 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:37.204493 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:38.208959 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:38.208923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" event={"ID":"1f2f1d1d-767b-4382-a94d-185d9dd269f2","Type":"ContainerStarted","Data":"0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5"} Jul 07 13:15:38.603251 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:38.603209 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:38.603493 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:38.603469 2578 patch_prober.go:28] interesting pod/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 13:15:38.603548 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:38.603529 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 13:15:39.603590 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:39.603551 2578 patch_prober.go:28] interesting pod/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 13:15:39.604013 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:39.603615 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 13:15:40.487751 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:40.487719 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp"] Jul 07 13:15:40.487973 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:40.487950 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" containerID="cri-o://88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e" gracePeriod=30 Jul 07 13:15:40.604169 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:40.604128 2578 patch_prober.go:28] interesting pod/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 13:15:40.604563 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:40.604198 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 13:15:41.604158 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:41.604119 2578 patch_prober.go:28] interesting pod/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 13:15:41.604353 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:41.604180 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 13:15:42.607564 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:42.607533 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:42.607965 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:42.607818 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:42.608621 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:42.608606 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:45.748258 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.748175 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:45.904715 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.904682 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-certs\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.904893 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.904725 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-credential-socket\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.904893 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.904741 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-data\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.904893 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.904790 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15598e31-9727-42e5-a48a-168e2e174b35-istiod-ca-cert\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.904893 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.904818 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-istio-token\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.905136 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.904972 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15598e31-9727-42e5-a48a-168e2e174b35-istio-podinfo\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.905136 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905000 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:45.905136 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905075 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:45.905136 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905096 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzmwt\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-kube-api-access-qzmwt\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.905136 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905131 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-socket\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.905373 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905164 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-envoy\") pod \"15598e31-9727-42e5-a48a-168e2e174b35\" (UID: \"15598e31-9727-42e5-a48a-168e2e174b35\") " Jul 07 13:15:45.905373 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905361 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:45.905557 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905536 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:45.905677 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905664 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:45.905780 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905766 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:45.905873 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905790 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15598e31-9727-42e5-a48a-168e2e174b35-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:45.906016 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.905950 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-data" (OuterVolumeSpecName: "istio-data") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:45.907315 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.907286 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:45.907438 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.907419 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-kube-api-access-qzmwt" (OuterVolumeSpecName: "kube-api-access-qzmwt") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "kube-api-access-qzmwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:45.907598 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.907572 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-istio-token" (OuterVolumeSpecName: "istio-token") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:45.907663 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:45.907591 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15598e31-9727-42e5-a48a-168e2e174b35-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15598e31-9727-42e5-a48a-168e2e174b35" (UID: "15598e31-9727-42e5-a48a-168e2e174b35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:15:46.007295 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.007208 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:46.007295 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.007235 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15598e31-9727-42e5-a48a-168e2e174b35-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:46.007295 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.007244 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15598e31-9727-42e5-a48a-168e2e174b35-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:46.007295 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.007254 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:46.007295 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.007261 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15598e31-9727-42e5-a48a-168e2e174b35-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:46.007295 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.007269 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzmwt\" (UniqueName: \"kubernetes.io/projected/15598e31-9727-42e5-a48a-168e2e174b35-kube-api-access-qzmwt\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:46.247788 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.247754 2578 generic.go:356] "Generic (PLEG): container finished" podID="15598e31-9727-42e5-a48a-168e2e174b35" containerID="88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e" exitCode=0 Jul 07 13:15:46.247934 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.247802 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" Jul 07 13:15:46.247934 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.247836 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" event={"ID":"15598e31-9727-42e5-a48a-168e2e174b35","Type":"ContainerDied","Data":"88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e"} Jul 07 13:15:46.247934 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.247892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp" event={"ID":"15598e31-9727-42e5-a48a-168e2e174b35","Type":"ContainerDied","Data":"179af93642dcd231f6f3a0aadfb4677700151b75e0995808c54d2cacb3ea87f5"} Jul 07 13:15:46.247934 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.247914 2578 scope.go:117] "RemoveContainer" containerID="88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e" Jul 07 13:15:46.257352 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.257310 2578 scope.go:117] "RemoveContainer" containerID="f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb" Jul 07 13:15:46.265319 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.265298 2578 scope.go:117] "RemoveContainer" containerID="88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e" Jul 07 13:15:46.265598 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:15:46.265579 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e\": container with ID starting with 88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e not found: ID does not exist" containerID="88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e" Jul 07 13:15:46.265649 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.265607 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e"} err="failed to get container status \"88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e\": rpc error: code = NotFound desc = could not find container \"88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e\": container with ID starting with 88cc455775be463f74be0f84c71ac9250465fb38c95f2671f985091a3f509a5e not found: ID does not exist" Jul 07 13:15:46.265649 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.265624 2578 scope.go:117] "RemoveContainer" containerID="f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb" Jul 07 13:15:46.265872 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:15:46.265854 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb\": container with ID starting with f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb not found: ID does not exist" containerID="f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb" Jul 07 13:15:46.265913 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.265878 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb"} err="failed to get container status \"f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb\": rpc error: code = NotFound desc = could not find container \"f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb\": container with ID starting with f7b55a6072e273d688e45601c70aded738f27f59a97c070001f44698e534ebfb not found: ID does not exist" Jul 07 13:15:46.271124 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.271100 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp"] Jul 07 13:15:46.277559 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:46.277535 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8ff4d1-openshift-default-79d79c47fc-vfwzp"] Jul 07 13:15:47.156553 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:47.156520 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15598e31-9727-42e5-a48a-168e2e174b35" path="/var/lib/kubelet/pods/15598e31-9727-42e5-a48a-168e2e174b35/volumes" Jul 07 13:15:47.692702 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:47.692665 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb"] Jul 07 13:15:47.692949 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:47.692924 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" containerID="cri-o://0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5" gracePeriod=30 Jul 07 13:15:48.650901 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.650868 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn"] Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651213 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651224 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651236 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651242 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651258 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651264 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a6d3c2-527b-4952-b427-d131c450cbe8" containerName="istio-proxy" Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651315 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" Jul 07 13:15:48.651336 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.651324 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="15598e31-9727-42e5-a48a-168e2e174b35" containerName="istio-proxy" Jul 07 13:15:48.654017 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.653997 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.660340 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.660312 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-0ebff7-openshift-default-dockercfg-tg25g\"" Jul 07 13:15:48.672591 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.672559 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn"] Jul 07 13:15:48.731976 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.731935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxxsk\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-kube-api-access-xxxsk\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732163 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-socket\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732163 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-credential-socket\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732163 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-envoy\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732319 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-certs\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732319 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732208 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-istio-token\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732319 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/733a4d96-69d9-457a-9e10-14d5761fc247-istio-podinfo\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732319 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732278 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/733a4d96-69d9-457a-9e10-14d5761fc247-istiod-ca-cert\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.732463 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.732371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-data\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.833472 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-envoy\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.833675 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833489 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-certs\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.833675 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-istio-token\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.833675 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/733a4d96-69d9-457a-9e10-14d5761fc247-istio-podinfo\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.833675 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/733a4d96-69d9-457a-9e10-14d5761fc247-istiod-ca-cert\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.833675 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-data\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.833675 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxxsk\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-kube-api-access-xxxsk\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.834008 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-socket\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.834008 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-credential-socket\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.834008 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.833967 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-certs\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.834187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.834025 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-socket\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.834187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.834084 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-data\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.834187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.834105 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-credential-socket\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.834381 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.834361 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/733a4d96-69d9-457a-9e10-14d5761fc247-istiod-ca-cert\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.835822 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.835794 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-envoy\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.836012 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.835994 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/733a4d96-69d9-457a-9e10-14d5761fc247-istio-podinfo\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.846562 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.846539 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-istio-token\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.846672 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.846657 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxxsk\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-kube-api-access-xxxsk\") pod \"e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:48.964474 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:48.964399 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:49.104505 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.104480 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn"] Jul 07 13:15:49.106635 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:15:49.106610 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733a4d96_69d9_457a_9e10_14d5761fc247.slice/crio-8d103ccb49dd4f9f77a867cbe4b8dae456098abff90c5871666af7f117bf3e96 WatchSource:0}: Error finding container 8d103ccb49dd4f9f77a867cbe4b8dae456098abff90c5871666af7f117bf3e96: Status 404 returned error can't find the container with id 8d103ccb49dd4f9f77a867cbe4b8dae456098abff90c5871666af7f117bf3e96 Jul 07 13:15:49.108781 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.108750 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:49.108883 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.108829 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:49.108883 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.108874 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:49.266563 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.266450 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" event={"ID":"733a4d96-69d9-457a-9e10-14d5761fc247","Type":"ContainerStarted","Data":"1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636"} Jul 07 13:15:49.266563 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.266528 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" event={"ID":"733a4d96-69d9-457a-9e10-14d5761fc247","Type":"ContainerStarted","Data":"8d103ccb49dd4f9f77a867cbe4b8dae456098abff90c5871666af7f117bf3e96"} Jul 07 13:15:49.296980 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.288822 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" podStartSLOduration=1.288799208 podStartE2EDuration="1.288799208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:49.287110932 +0000 UTC m=+2022.631765123" watchObservedRunningTime="2026-07-07 13:15:49.288799208 +0000 UTC m=+2022.633453393" Jul 07 13:15:49.965135 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:49.965090 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:50.969746 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:50.969713 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:51.276517 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:51.276424 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:51.277330 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:51.277312 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:15:52.941264 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.941238 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:52.967156 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967077 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-token\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967156 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967120 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-data\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967386 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967168 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-podinfo\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967459 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967432 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-credential-socket\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967518 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967490 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istiod-ca-cert\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967579 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967562 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-certs\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967635 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967585 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-socket\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967635 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967617 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-envoy\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967742 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967680 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z47s\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-kube-api-access-6z47s\") pod \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\" (UID: \"1f2f1d1d-767b-4382-a94d-185d9dd269f2\") " Jul 07 13:15:52.967742 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967696 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:52.967939 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.967899 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:52.968309 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.968022 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:52.968697 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.968652 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:15:52.968697 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.968638 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:52.968838 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.968756 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:52.969852 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.969823 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:15:52.970490 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.970471 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:52.970956 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.970938 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:15:52.971011 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:52.970939 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-kube-api-access-6z47s" (OuterVolumeSpecName: "kube-api-access-6z47s") pod "1f2f1d1d-767b-4382-a94d-185d9dd269f2" (UID: "1f2f1d1d-767b-4382-a94d-185d9dd269f2"). InnerVolumeSpecName "kube-api-access-6z47s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:15:53.068914 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068872 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z47s\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-kube-api-access-6z47s\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.068914 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068907 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.068914 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068918 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.068914 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068926 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.069221 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068934 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.069221 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068943 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.069221 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068951 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.069221 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.068958 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f2f1d1d-767b-4382-a94d-185d9dd269f2-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:15:53.286776 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.286683 2578 generic.go:356] "Generic (PLEG): container finished" podID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerID="0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5" exitCode=0 Jul 07 13:15:53.286776 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.286731 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" Jul 07 13:15:53.287021 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.286771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" event={"ID":"1f2f1d1d-767b-4382-a94d-185d9dd269f2","Type":"ContainerDied","Data":"0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5"} Jul 07 13:15:53.287021 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.286835 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb" event={"ID":"1f2f1d1d-767b-4382-a94d-185d9dd269f2","Type":"ContainerDied","Data":"545e8b8cc791ec506c31e83bdf3f75574edb62164fde6494072d8882913488ad"} Jul 07 13:15:53.287021 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.286864 2578 scope.go:117] "RemoveContainer" containerID="0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5" Jul 07 13:15:53.297709 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.297680 2578 scope.go:117] "RemoveContainer" containerID="2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585" Jul 07 13:15:53.306202 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.306175 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb"] Jul 07 13:15:53.308102 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.308084 2578 scope.go:117] "RemoveContainer" containerID="0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5" Jul 07 13:15:53.308498 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:15:53.308475 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5\": container with ID starting with 0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5 not found: ID does not exist" containerID="0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5" Jul 07 13:15:53.308658 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.308505 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5"} err="failed to get container status \"0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5\": rpc error: code = NotFound desc = could not find container \"0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5\": container with ID starting with 0c71df731f0de196e3069d9c3f04ba18c2e7ded847a29a63c23b84ce7efdbca5 not found: ID does not exist" Jul 07 13:15:53.308658 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.308523 2578 scope.go:117] "RemoveContainer" containerID="2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585" Jul 07 13:15:53.308845 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:15:53.308828 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585\": container with ID starting with 2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585 not found: ID does not exist" containerID="2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585" Jul 07 13:15:53.308905 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.308849 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585"} err="failed to get container status \"2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585\": rpc error: code = NotFound desc = could not find container \"2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585\": container with ID starting with 2d7bf9f3145bd638f6f096cba8c5dbd35c4dccfe8ab0cc0a2222f5a9e00e3585 not found: ID does not exist" Jul 07 13:15:53.309685 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:53.309663 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8e00d7-openshift-default-7bf97d49-8wrqb"] Jul 07 13:15:55.156930 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:55.156883 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" path="/var/lib/kubelet/pods/1f2f1d1d-767b-4382-a94d-185d9dd269f2/volumes" Jul 07 13:15:56.399497 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.399461 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b"] Jul 07 13:15:56.399981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.399805 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" Jul 07 13:15:56.399981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.399816 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" Jul 07 13:15:56.399981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.399827 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" Jul 07 13:15:56.399981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.399833 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" Jul 07 13:15:56.399981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.399899 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" Jul 07 13:15:56.399981 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.399909 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2f1d1d-767b-4382-a94d-185d9dd269f2" containerName="istio-proxy" Jul 07 13:15:56.403592 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.403570 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.407176 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.407156 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e80c89-openshift-default-dockercfg-jzcw2\"" Jul 07 13:15:56.418080 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.418057 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b"] Jul 07 13:15:56.499533 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499493 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-envoy\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.499533 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vd6v\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-kube-api-access-2vd6v\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.499777 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499628 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istiod-ca-cert\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.499777 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-podinfo\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.499777 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-token\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.499942 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-data\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.499942 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-credential-socket\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.499942 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-socket\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.500075 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.499989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-certs\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.600969 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.600936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-envoy\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.600969 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.600975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vd6v\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-kube-api-access-2vd6v\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601244 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istiod-ca-cert\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601244 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-podinfo\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601244 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601161 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-token\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601244 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-data\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601244 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-credential-socket\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601515 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-socket\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601515 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-certs\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601629 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601609 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-data\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601687 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601659 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-certs\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601745 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601689 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-credential-socket\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601812 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601727 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-socket\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.601941 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.601920 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istiod-ca-cert\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.603831 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.603814 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-envoy\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.604164 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.604141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-podinfo\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.609716 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.609681 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vd6v\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-kube-api-access-2vd6v\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.611138 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.611116 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-token\") pod \"e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.713715 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.713604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:56.850343 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.850320 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b"] Jul 07 13:15:56.852499 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:15:56.852468 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10808d20_7fc7_4ef9_b4f1_3ab28aa71626.slice/crio-baf74ff35be051a3a3b3188a069e3e17cde4418d8a0910526c94a2fa7663170a WatchSource:0}: Error finding container baf74ff35be051a3a3b3188a069e3e17cde4418d8a0910526c94a2fa7663170a: Status 404 returned error can't find the container with id baf74ff35be051a3a3b3188a069e3e17cde4418d8a0910526c94a2fa7663170a Jul 07 13:15:56.854655 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.854615 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:56.854757 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.854706 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:56.854757 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:56.854745 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:15:57.307539 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:57.307498 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" event={"ID":"10808d20-7fc7-4ef9-b4f1-3ab28aa71626","Type":"ContainerStarted","Data":"dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9"} Jul 07 13:15:57.307732 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:57.307561 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" event={"ID":"10808d20-7fc7-4ef9-b4f1-3ab28aa71626","Type":"ContainerStarted","Data":"baf74ff35be051a3a3b3188a069e3e17cde4418d8a0910526c94a2fa7663170a"} Jul 07 13:15:57.332623 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:57.332562 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" podStartSLOduration=1.332543764 podStartE2EDuration="1.332543764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:15:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:15:57.331306688 +0000 UTC m=+2030.675960884" watchObservedRunningTime="2026-07-07 13:15:57.332543764 +0000 UTC m=+2030.677197947" Jul 07 13:15:57.714068 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:57.714010 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:57.719174 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:57.719139 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:58.312519 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:58.312489 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:15:58.313390 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:15:58.313367 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:16:06.349197 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:06.346837 2578 patch_prober.go:28] interesting pod/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": read tcp 10.132.0.2:37950->10.132.0.39:15021: read: connection reset by peer" start-of-body= Jul 07 13:16:06.349197 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:06.346939 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": read tcp 10.132.0.2:37950->10.132.0.39:15021: read: connection reset by peer" Jul 07 13:16:07.370543 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:07.370509 2578 generic.go:356] "Generic (PLEG): container finished" podID="733a4d96-69d9-457a-9e10-14d5761fc247" containerID="1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636" exitCode=0 Jul 07 13:16:07.371076 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:07.370587 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" event={"ID":"733a4d96-69d9-457a-9e10-14d5761fc247","Type":"ContainerDied","Data":"1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636"} Jul 07 13:16:07.371169 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:07.371150 2578 scope.go:117] "RemoveContainer" containerID="1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636" Jul 07 13:16:07.372211 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:07.372183 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:16:07.372335 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:07.372259 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:16:07.372335 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:07.372295 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:16:08.384912 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:08.384864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" event={"ID":"733a4d96-69d9-457a-9e10-14d5761fc247","Type":"ContainerStarted","Data":"164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af"} Jul 07 13:16:08.966010 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:08.965899 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:16:08.966010 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:08.965978 2578 patch_prober.go:28] interesting pod/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 13:16:08.966344 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:08.966036 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 13:16:09.965574 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:09.965535 2578 patch_prober.go:28] interesting pod/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 13:16:09.966275 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:09.966242 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 13:16:10.965169 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:10.965138 2578 patch_prober.go:28] interesting pod/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 13:16:10.965339 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:10.965208 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 13:16:11.965558 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:11.965520 2578 patch_prober.go:28] interesting pod/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 07 13:16:11.966033 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:11.965580 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 07 13:16:12.968995 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:12.968962 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:16:12.969440 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:12.969419 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:16:12.970133 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:12.970115 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:16:13.627702 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:13.627402 2578 patch_prober.go:28] interesting pod/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:34642->10.132.0.40:15021: read: connection reset by peer" start-of-body= Jul 07 13:16:13.627702 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:13.627459 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:34642->10.132.0.40:15021: read: connection reset by peer" Jul 07 13:16:14.414362 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:14.414328 2578 generic.go:356] "Generic (PLEG): container finished" podID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerID="dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9" exitCode=0 Jul 07 13:16:14.414758 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:14.414399 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" event={"ID":"10808d20-7fc7-4ef9-b4f1-3ab28aa71626","Type":"ContainerDied","Data":"dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9"} Jul 07 13:16:14.414871 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:14.414852 2578 scope.go:117] "RemoveContainer" containerID="dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9" Jul 07 13:16:14.415837 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:14.415811 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:16:14.415940 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:14.415887 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:16:14.416005 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:14.415937 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:16:15.421793 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:15.421759 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" event={"ID":"10808d20-7fc7-4ef9-b4f1-3ab28aa71626","Type":"ContainerStarted","Data":"63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff"} Jul 07 13:16:15.714011 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:15.713924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:16:15.714193 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:15.714151 2578 patch_prober.go:28] interesting pod/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 13:16:15.714236 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:15.714211 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 13:16:16.714760 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:16.714723 2578 patch_prober.go:28] interesting pod/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 13:16:16.715179 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:16.714786 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 13:16:17.714149 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:17.714109 2578 patch_prober.go:28] interesting pod/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 13:16:17.714344 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:17.714167 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 13:16:18.714635 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:18.714578 2578 patch_prober.go:28] interesting pod/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 13:16:18.715079 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:18.714673 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 13:16:19.717989 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:19.717954 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:16:19.718498 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:19.718365 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:16:19.718932 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:16:19.718913 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:17:07.189602 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:17:07.189573 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:17:07.194196 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:17:07.194171 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:22:03.116257 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:03.116211 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn"] Jul 07 13:22:03.116738 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:03.116451 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" containerID="cri-o://164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af" gracePeriod=30 Jul 07 13:22:07.216030 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:07.215997 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:22:07.237468 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:07.237434 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:22:08.368029 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.368000 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:22:08.540015 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.539906 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-credential-socket\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540015 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.539995 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/733a4d96-69d9-457a-9e10-14d5761fc247-istio-podinfo\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540015 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540016 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/733a4d96-69d9-457a-9e10-14d5761fc247-istiod-ca-cert\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540368 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540074 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-socket\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540368 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540101 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-certs\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540368 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540150 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxxsk\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-kube-api-access-xxxsk\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540368 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540174 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-data\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540368 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540225 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-envoy\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540368 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540280 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-istio-token\") pod \"733a4d96-69d9-457a-9e10-14d5761fc247\" (UID: \"733a4d96-69d9-457a-9e10-14d5761fc247\") " Jul 07 13:22:08.540368 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540335 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:08.540750 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540419 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:08.540750 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540591 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.540750 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540611 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.540750 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540618 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/733a4d96-69d9-457a-9e10-14d5761fc247-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:08.540953 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.540754 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:08.541101 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.541077 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-data" (OuterVolumeSpecName: "istio-data") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:08.542472 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.542439 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:08.542472 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.542453 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-istio-token" (OuterVolumeSpecName: "istio-token") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:08.542614 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.542535 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/733a4d96-69d9-457a-9e10-14d5761fc247-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:22:08.542614 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.542552 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-kube-api-access-xxxsk" (OuterVolumeSpecName: "kube-api-access-xxxsk") pod "733a4d96-69d9-457a-9e10-14d5761fc247" (UID: "733a4d96-69d9-457a-9e10-14d5761fc247"). InnerVolumeSpecName "kube-api-access-xxxsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:08.641145 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.641106 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxxsk\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-kube-api-access-xxxsk\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.641145 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.641137 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.641145 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.641146 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.641145 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.641155 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/733a4d96-69d9-457a-9e10-14d5761fc247-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.641431 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.641162 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/733a4d96-69d9-457a-9e10-14d5761fc247-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.641431 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.641171 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/733a4d96-69d9-457a-9e10-14d5761fc247-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:08.641431 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:08.641178 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/733a4d96-69d9-457a-9e10-14d5761fc247-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:09.064513 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.064477 2578 generic.go:356] "Generic (PLEG): container finished" podID="733a4d96-69d9-457a-9e10-14d5761fc247" containerID="164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af" exitCode=0 Jul 07 13:22:09.064712 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.064530 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" Jul 07 13:22:09.064712 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.064562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" event={"ID":"733a4d96-69d9-457a-9e10-14d5761fc247","Type":"ContainerDied","Data":"164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af"} Jul 07 13:22:09.064712 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.064641 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn" event={"ID":"733a4d96-69d9-457a-9e10-14d5761fc247","Type":"ContainerDied","Data":"8d103ccb49dd4f9f77a867cbe4b8dae456098abff90c5871666af7f117bf3e96"} Jul 07 13:22:09.064712 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.064664 2578 scope.go:117] "RemoveContainer" containerID="164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af" Jul 07 13:22:09.076322 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.076298 2578 scope.go:117] "RemoveContainer" containerID="1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636" Jul 07 13:22:09.085250 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.085230 2578 scope.go:117] "RemoveContainer" containerID="164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af" Jul 07 13:22:09.085602 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:22:09.085577 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af\": container with ID starting with 164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af not found: ID does not exist" containerID="164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af" Jul 07 13:22:09.085650 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.085613 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af"} err="failed to get container status \"164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af\": rpc error: code = NotFound desc = could not find container \"164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af\": container with ID starting with 164a8a0ea81ec2ebf4681b8bd7482cf1c8508621516f0e91c96c4673c84ba5af not found: ID does not exist" Jul 07 13:22:09.085650 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.085632 2578 scope.go:117] "RemoveContainer" containerID="1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636" Jul 07 13:22:09.085935 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:22:09.085913 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636\": container with ID starting with 1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636 not found: ID does not exist" containerID="1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636" Jul 07 13:22:09.086011 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.085939 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636"} err="failed to get container status \"1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636\": rpc error: code = NotFound desc = could not find container \"1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636\": container with ID starting with 1658a5db74a61825e0ca0eb80ed761314e0a5596d687fbed40f4ee8854e08636 not found: ID does not exist" Jul 07 13:22:09.092130 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.092102 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn"] Jul 07 13:22:09.095127 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.095025 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-0ebff7-openshift-default-574b9d4cff-h6xsn"] Jul 07 13:22:09.157650 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:09.157618 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" path="/var/lib/kubelet/pods/733a4d96-69d9-457a-9e10-14d5761fc247/volumes" Jul 07 13:22:10.500622 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:10.500590 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b"] Jul 07 13:22:10.501072 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:10.500806 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" containerID="cri-o://63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff" gracePeriod=30 Jul 07 13:22:15.756824 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.756795 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:22:15.906486 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906454 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-data\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906486 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906494 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-credential-socket\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906523 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-socket\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906596 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vd6v\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-kube-api-access-2vd6v\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906619 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istiod-ca-cert\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906642 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-podinfo\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906664 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-certs\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906691 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-envoy\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.906766 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906726 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-token\") pod \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\" (UID: \"10808d20-7fc7-4ef9-b4f1-3ab28aa71626\") " Jul 07 13:22:15.907166 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906786 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:15.907166 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.906800 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:15.907166 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.907070 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:15.907340 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.907167 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-credential-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:15.907340 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.907206 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-socket\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:15.907340 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.907284 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-data" (OuterVolumeSpecName: "istio-data") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:15.907340 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.907305 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:22:15.908947 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.908918 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:22:15.909102 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.908983 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:22:15.909102 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.909019 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-token" (OuterVolumeSpecName: "istio-token") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:15.909102 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:15.909032 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-kube-api-access-2vd6v" (OuterVolumeSpecName: "kube-api-access-2vd6v") pod "10808d20-7fc7-4ef9-b4f1-3ab28aa71626" (UID: "10808d20-7fc7-4ef9-b4f1-3ab28aa71626"). InnerVolumeSpecName "kube-api-access-2vd6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:22:16.007935 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.007890 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-data\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:16.007935 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.007930 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vd6v\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-kube-api-access-2vd6v\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:16.007935 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.007942 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istiod-ca-cert\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:16.007935 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.007951 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-podinfo\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:16.008267 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.007959 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-workload-certs\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:16.008267 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.007967 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-envoy\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:16.008267 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.007974 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10808d20-7fc7-4ef9-b4f1-3ab28aa71626-istio-token\") on node \"ip-10-0-141-76.ec2.internal\" DevicePath \"\"" Jul 07 13:22:16.100314 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.100277 2578 generic.go:356] "Generic (PLEG): container finished" podID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerID="63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff" exitCode=0 Jul 07 13:22:16.100467 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.100325 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" Jul 07 13:22:16.100467 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.100355 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" event={"ID":"10808d20-7fc7-4ef9-b4f1-3ab28aa71626","Type":"ContainerDied","Data":"63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff"} Jul 07 13:22:16.100467 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.100407 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b" event={"ID":"10808d20-7fc7-4ef9-b4f1-3ab28aa71626","Type":"ContainerDied","Data":"baf74ff35be051a3a3b3188a069e3e17cde4418d8a0910526c94a2fa7663170a"} Jul 07 13:22:16.100467 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.100429 2578 scope.go:117] "RemoveContainer" containerID="63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff" Jul 07 13:22:16.109897 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.109877 2578 scope.go:117] "RemoveContainer" containerID="dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9" Jul 07 13:22:16.118616 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.118592 2578 scope.go:117] "RemoveContainer" containerID="63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff" Jul 07 13:22:16.118932 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:22:16.118912 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff\": container with ID starting with 63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff not found: ID does not exist" containerID="63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff" Jul 07 13:22:16.118997 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.118947 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff"} err="failed to get container status \"63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff\": rpc error: code = NotFound desc = could not find container \"63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff\": container with ID starting with 63c758b720ab5c30edc1b878b4b2490b28d462eaca1f85af67f6deb2379caeff not found: ID does not exist" Jul 07 13:22:16.118997 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.118973 2578 scope.go:117] "RemoveContainer" containerID="dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9" Jul 07 13:22:16.119242 ip-10-0-141-76 kubenswrapper[2578]: E0707 13:22:16.119220 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9\": container with ID starting with dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9 not found: ID does not exist" containerID="dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9" Jul 07 13:22:16.119296 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.119250 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9"} err="failed to get container status \"dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9\": rpc error: code = NotFound desc = could not find container \"dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9\": container with ID starting with dd152a1eb3edb0d1ca422eecec7a29acf04e1e305fc72991e60c33611df950f9 not found: ID does not exist" Jul 07 13:22:16.122929 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.122897 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b"] Jul 07 13:22:16.129166 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.129140 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e80c89-openshift-default-66b4888695-7bx5b"] Jul 07 13:22:16.808149 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.808119 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-rgm4b_a44e2e6b-06a6-4cab-b490-783fdab05203/copy-manifests/0.log" Jul 07 13:22:16.815997 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:16.815967 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-rgm4b_a44e2e6b-06a6-4cab-b490-783fdab05203/manager/0.log" Jul 07 13:22:17.156485 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:17.156443 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" path="/var/lib/kubelet/pods/10808d20-7fc7-4ef9-b4f1-3ab28aa71626/volumes" Jul 07 13:22:19.659680 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:19.659634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45vvd5_bbf978ee-cbd1-4d1b-af76-e2045dd92a32/istio-proxy/0.log" Jul 07 13:22:19.792600 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:19.792554 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bnjzp_4becf08f-fc61-4413-b9da-3ce0d7c91df8/discovery/0.log" Jul 07 13:22:20.539832 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:20.539801 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67ff897d47-8lqlg_b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7/router/0.log" Jul 07 13:22:20.910130 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:20.910097 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8smgn_ac2e12c7-a853-4efa-a3b0-00a02147ff11/main/0.log" Jul 07 13:22:21.268367 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:21.268272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbghx9z_3d5fb75f-f0d2-4bc8-8661-39e13d32923c/main/0.log" Jul 07 13:22:21.388880 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:21.388848 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-w28b9_9df3024b-e63c-4bbb-95b2-d7fb3f309f42/main/0.log" Jul 07 13:22:29.037504 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:29.037473 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-22mxz_0cc952fb-050d-439d-a39a-3167617f0640/global-pull-secret-syncer/0.log" Jul 07 13:22:29.573141 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:29.573109 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-72ltc_4df33d13-2cae-46fc-a03f-e104124d7d38/konnectivity-agent/0.log" Jul 07 13:22:29.688474 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:29.688442 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-76.ec2.internal_764707e362efae25fa03b75c43299a59/haproxy/0.log" Jul 07 13:22:36.271317 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:36.271274 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-6llv9_1acc0eb6-6234-43f8-ae0e-fda54a04ce53/cluster-monitoring-operator/0.log" Jul 07 13:22:36.451620 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:36.451589 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95sts_14d0298b-2c83-4e97-8d37-36588e79000f/node-exporter/0.log" Jul 07 13:22:36.480163 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:36.480134 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95sts_14d0298b-2c83-4e97-8d37-36588e79000f/kube-rbac-proxy/0.log" Jul 07 13:22:36.514345 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:36.514317 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95sts_14d0298b-2c83-4e97-8d37-36588e79000f/init-textfile/0.log" Jul 07 13:22:38.562823 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:38.562790 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-67776_f1c22b51-fd94-4663-86d4-53c79d11996e/networking-console-plugin/0.log" Jul 07 13:22:39.238833 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.238793 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh"] Jul 07 13:22:39.239167 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239131 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" Jul 07 13:22:39.239167 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239148 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" Jul 07 13:22:39.239167 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239159 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" Jul 07 13:22:39.239167 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239164 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" Jul 07 13:22:39.239407 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239175 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" Jul 07 13:22:39.239407 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239180 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" Jul 07 13:22:39.239407 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239190 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" Jul 07 13:22:39.239407 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239195 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" Jul 07 13:22:39.239407 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239241 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" Jul 07 13:22:39.239407 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239249 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="733a4d96-69d9-457a-9e10-14d5761fc247" containerName="istio-proxy" Jul 07 13:22:39.239407 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.239255 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="10808d20-7fc7-4ef9-b4f1-3ab28aa71626" containerName="istio-proxy" Jul 07 13:22:39.242235 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.242212 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.249377 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.249348 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gwgv6\"/\"openshift-service-ca.crt\"" Jul 07 13:22:39.249513 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.249389 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gwgv6\"/\"kube-root-ca.crt\"" Jul 07 13:22:39.249513 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.249348 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gwgv6\"/\"default-dockercfg-65jtm\"" Jul 07 13:22:39.253609 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.253579 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh"] Jul 07 13:22:39.272991 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.272959 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/1.log" Jul 07 13:22:39.282310 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.282274 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-d67d7_0430f99a-b20d-4fb2-b994-1d150ef9cb30/console-operator/2.log" Jul 07 13:22:39.308479 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.308437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-sys\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.308479 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.308480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t94t\" (UniqueName: \"kubernetes.io/projected/07370887-c8a4-49c4-b5dd-f8581849d4c8-kube-api-access-4t94t\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.308700 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.308501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-podres\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.308700 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.308561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-proc\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.308700 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.308626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-lib-modules\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.409966 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.409925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-lib-modules\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.409980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-sys\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.410099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t94t\" (UniqueName: \"kubernetes.io/projected/07370887-c8a4-49c4-b5dd-f8581849d4c8-kube-api-access-4t94t\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.410146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-podres\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.410166 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-lib-modules\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410187 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.410188 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-sys\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410449 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.410191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-proc\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410449 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.410237 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-proc\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.410449 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.410262 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07370887-c8a4-49c4-b5dd-f8581849d4c8-podres\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.424029 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.423991 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t94t\" (UniqueName: \"kubernetes.io/projected/07370887-c8a4-49c4-b5dd-f8581849d4c8-kube-api-access-4t94t\") pod \"perf-node-gather-daemonset-tnglh\" (UID: \"07370887-c8a4-49c4-b5dd-f8581849d4c8\") " pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.553559 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.553442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:39.690615 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:39.690588 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh"] Jul 07 13:22:39.692677 ip-10-0-141-76 kubenswrapper[2578]: W0707 13:22:39.692648 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod07370887_c8a4_49c4_b5dd_f8581849d4c8.slice/crio-5031b52d00206d625ecb0f465cfe196aef1a268dd1ee7cbb0192651d7edb819c WatchSource:0}: Error finding container 5031b52d00206d625ecb0f465cfe196aef1a268dd1ee7cbb0192651d7edb819c: Status 404 returned error can't find the container with id 5031b52d00206d625ecb0f465cfe196aef1a268dd1ee7cbb0192651d7edb819c Jul 07 13:22:40.211698 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:40.211655 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" event={"ID":"07370887-c8a4-49c4-b5dd-f8581849d4c8","Type":"ContainerStarted","Data":"18ea2ad627a9f84e3b538da6f5e8966f39f44fffc2c62bad0308c872d6994499"} Jul 07 13:22:40.211891 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:40.211709 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" event={"ID":"07370887-c8a4-49c4-b5dd-f8581849d4c8","Type":"ContainerStarted","Data":"5031b52d00206d625ecb0f465cfe196aef1a268dd1ee7cbb0192651d7edb819c"} Jul 07 13:22:40.211891 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:40.211796 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:40.231420 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:40.231362 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" podStartSLOduration=1.231343092 podStartE2EDuration="1.231343092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:40.23040448 +0000 UTC m=+2433.575058665" watchObservedRunningTime="2026-07-07 13:22:40.231343092 +0000 UTC m=+2433.575997276" Jul 07 13:22:40.616909 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:40.616880 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-nlgs8_d3580b95-a6d3-4838-bac6-5c65e87f1760/volume-data-source-validator/0.log" Jul 07 13:22:41.679104 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:41.679056 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4xbtp_d2e4302f-b012-41ab-afc0-28c1f36f5389/dns/0.log" Jul 07 13:22:41.704445 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:41.704413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4xbtp_d2e4302f-b012-41ab-afc0-28c1f36f5389/kube-rbac-proxy/0.log" Jul 07 13:22:41.876011 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:41.875976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lwwtx_71c2d709-99a0-43a4-82c0-632eed5c966a/dns-node-resolver/0.log" Jul 07 13:22:42.494624 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:42.494593 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s27kt_3ac46ca0-b6cf-45d8-88ca-cd0553738b14/node-ca/0.log" Jul 07 13:22:43.482403 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:43.482371 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45vvd5_bbf978ee-cbd1-4d1b-af76-e2045dd92a32/istio-proxy/0.log" Jul 07 13:22:43.675612 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:43.675579 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bnjzp_4becf08f-fc61-4413-b9da-3ce0d7c91df8/discovery/0.log" Jul 07 13:22:43.902235 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:43.902199 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67ff897d47-8lqlg_b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7/router/0.log" Jul 07 13:22:44.570387 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:44.570352 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-26jzv_05a7a66a-0fe9-4c97-a899-d5c914212263/serve-healthcheck-canary/0.log" Jul 07 13:22:45.133816 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:45.133778 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ckz8l_53e4e2b2-0715-449f-b683-43191049e45a/insights-operator/0.log" Jul 07 13:22:45.134723 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:45.134704 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ckz8l_53e4e2b2-0715-449f-b683-43191049e45a/insights-operator/1.log" Jul 07 13:22:45.256029 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:45.255997 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ht647_f96a07f6-2590-4338-8df3-4cfe79d22b4c/kube-rbac-proxy/0.log" Jul 07 13:22:45.287297 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:45.287270 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ht647_f96a07f6-2590-4338-8df3-4cfe79d22b4c/exporter/0.log" Jul 07 13:22:45.313991 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:45.313966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ht647_f96a07f6-2590-4338-8df3-4cfe79d22b4c/extractor/0.log" Jul 07 13:22:46.227208 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:46.227180 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gwgv6/perf-node-gather-daemonset-tnglh" Jul 07 13:22:48.036009 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:48.035977 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-rgm4b_a44e2e6b-06a6-4cab-b490-783fdab05203/manager/0.log" Jul 07 13:22:48.060870 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:48.060842 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68cfd86456-rgm4b_a44e2e6b-06a6-4cab-b490-783fdab05203/copy-manifests/0.log" Jul 07 13:22:49.552084 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:49.552033 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65fd447f77-hsmvg_465fb2a0-a462-4f50-a903-6ea31e46c4e7/manager/0.log" Jul 07 13:22:49.682111 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:49.681997 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45vvd5_bbf978ee-cbd1-4d1b-af76-e2045dd92a32/istio-proxy/0.log" Jul 07 13:22:49.878724 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:49.878690 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bnjzp_4becf08f-fc61-4413-b9da-3ce0d7c91df8/discovery/0.log" Jul 07 13:22:50.032239 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:50.032133 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67ff897d47-8lqlg_b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7/router/0.log" Jul 07 13:22:51.241467 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:51.241386 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45vvd5_bbf978ee-cbd1-4d1b-af76-e2045dd92a32/istio-proxy/0.log" Jul 07 13:22:51.459249 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:51.459213 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bnjzp_4becf08f-fc61-4413-b9da-3ce0d7c91df8/discovery/0.log" Jul 07 13:22:51.702022 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:51.701990 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67ff897d47-8lqlg_b910b0e1-0a3f-48aa-82d4-f7f5373eb9c7/router/0.log" Jul 07 13:22:56.087711 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:56.087678 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-dkkf8_09307f15-ddb7-4d32-9af7-6021f032283e/kube-storage-version-migrator-operator/1.log" Jul 07 13:22:56.089136 ip-10-0-141-76 kubenswrapper[2578]: I0707 13:22:56.089109 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-dkkf8_09307f15-ddb7-4d32-9af7-6021f032283e/kube-storage-version-migrator-operator/0.log"